-- Logs begin at Tue 2022-06-07 13:30:48 UTC, end at Tue 2022-06-07 14:43:11 UTC. -- Jun 07 14:06:20.259143 dv-cinder-836083-9-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 07 14:06:20.266529 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 07 14:06:20.268846 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jun 7 14:06:20 2022] *** Jun 07 14:06:20.268989 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 07 14:06:20.269080 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 07 14:06:20.269168 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: nodename: dv-cinder-836083-9-iscsi Jun 07 14:06:20.269252 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: machine: x86_64 Jun 07 14:06:20.270912 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: clock source: unix Jun 07 14:06:20.270912 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: pcre jit disabled Jun 07 14:06:20.270912 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: detected number of CPU cores: 4 Jun 07 14:06:20.270912 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: current working directory: / Jun 07 14:06:20.270912 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: detected binary path: /usr/bin/uwsgi-core Jun 07 14:06:20.270912 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: your processes number limit is 47680 Jun 07 14:06:20.270912 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: your memory page size is 4096 bytes Jun 07 14:06:20.270912 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: detected max file descriptor number: 2048 Jun 07 14:06:20.270912 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: lock engine: pthread robust mutexes Jun 07 14:06:20.270912 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: thunder lock: enabled Jun 07 14:06:20.270912 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 07 14:06:20.270912 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 07 14:06:20.315481 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: Python main interpreter initialized at 0x5636b57729c0 Jun 07 14:06:20.315481 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: python threads support enabled Jun 07 14:06:20.315481 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: your server socket listen backlog is limited to 100 connections Jun 07 14:06:20.315481 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: your mercy for graceful operations on workers is 90 seconds Jun 07 14:06:20.315857 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: mapped 403077 bytes (393 KB) for 2 cores Jun 07 14:06:20.315936 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: *** Operational MODE: preforking *** Jun 07 14:06:20.316169 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: *** uWSGI is running in multiple interpreter mode *** Jun 07 14:06:20.316169 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: spawned uWSGI master process (pid: 80223) Jun 07 14:06:20.316378 dv-cinder-836083-9-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 07 14:06:20.318843 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: spawned uWSGI worker 1 (pid: 80233, cores: 1) Jun 07 14:06:20.318843 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: spawned uWSGI worker 2 (pid: 80234, cores: 1) Jun 07 14:06:20.318843 dv-cinder-836083-9-iscsi devstack@keystone.service[80223]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 07 14:06:26.078828 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 14:06:26.078828 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: from cryptography.utils import int_from_bytes Jun 07 14:06:26.079668 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 14:06:26.079668 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: from cryptography.utils import int_from_bytes Jun 07 14:06:26.080684 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 14:06:26.080684 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: from cryptography.utils import int_from_bytes Jun 07 14:06:26.081139 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 14:06:26.081139 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: from cryptography.utils import int_from_bytes Jun 07 14:06:26.130112 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 07 14:06:26.130315 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 07 14:06:26.130508 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] command line args: [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 07 14:06:26.130781 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 07 14:06:26.130961 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ================================================================================ {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 07 14:06:26.131048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 07 14:06:26.131236 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 07 14:06:26.131325 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] admin_token = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.131428 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] command line args: [] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 07 14:06:26.131506 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] config_dir = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.131662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.131745 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 07 14:06:26.131844 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] config_source = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.131925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ================================================================================ {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 07 14:06:26.132025 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] debug = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.132102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] admin_token = **** {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.132224 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.132519 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] config_dir = [] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.132519 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.132716 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.132810 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] config_source = [] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.132911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.133010 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] debug = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.133127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.133232 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.133232 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.133449 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.133556 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.133655 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.133655 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_config_append = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.133826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.133918 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.134026 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.134127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_dir = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.134221 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] list_limit = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.134313 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_file = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.134439 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] log_config_append = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.134568 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.134651 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.134651 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] log_dir = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.134825 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.134942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] log_file = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.135016 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.135133 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.135245 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.135324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.135438 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.135438 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.135671 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.135843 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.135964 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.136051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.136129 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.136244 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.136323 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.136402 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.136515 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.136589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.136749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.136865 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.136985 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.137071 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.137220 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.137345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.137469 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.137576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.137687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.137911 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.138032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.81/identity {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.138150 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.138260 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] publish_errors = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.138382 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.138497 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.138614 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.138769 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.138881 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.81/identity {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.139000 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.139111 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] publish_errors = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.139246 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.139354 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.139473 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] standard_threads = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.139648 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.139771 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.139881 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.139991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.140111 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.140111 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.140325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.140437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] use_journal = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.140437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] use_json = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.140634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] standard_threads = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.140751 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] use_stderr = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.140863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.140979 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] use_syslog = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.141090 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.141200 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.141310 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.141428 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.141546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] use_journal = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.141546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] use_json = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.141760 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.141870 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] use_stderr = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.141985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.142104 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] use_syslog = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.142216 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.142332 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:06:26.142449 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.142566 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.142677 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.142796 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.142907 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.143023 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.143133 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.143266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.143384 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.143570 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.143685 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.143801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.143801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.144013 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.144125 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.144242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.144353 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.144464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.144574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.144684 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.144794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.144912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.145030 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.145145 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.145256 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.145366 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.145482 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.145604 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.145722 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.145834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.145944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.146059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.146170 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.146291 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.146408 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.146519 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.146635 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.146747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.146862 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.146983 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.147094 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.147209 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.147348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.147458 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.147581 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.147694 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.147805 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.147916 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.148026 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.148137 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.148255 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.148367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.148477 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.148587 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.148703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.148821 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.149011 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.149132 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.149243 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.149353 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.149464 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.149574 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.149692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.149804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.149921 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.150032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.150133 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.150242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.150360 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.150472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.150583 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.150693 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.150792 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.150885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.150987 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.151082 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.151180 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.151859 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.151962 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.152056 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.152169 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.152265 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.152265 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.152446 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.152546 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.152640 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.152741 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.152835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.152933 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.153033 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.153127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.153220 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.153349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.153452 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.153600 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.153708 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.153829 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.153934 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.154066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.154175 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.154305 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.154411 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.154512 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.154613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.154752 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.154867 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.154970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.155084 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.155202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.155346 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.155474 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.155586 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.155698 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.155808 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.155909 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.156020 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.156133 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.156256 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.156361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.156479 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.156591 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.156703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.156814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.156933 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.157045 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.157163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.157288 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.157413 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.157531 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.157654 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.157780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.157897 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.158014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.158127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.158239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.158358 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.158471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.158583 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.158694 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.158812 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.158925 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.159044 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.159157 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.159320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.159435 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.159547 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.159658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.159782 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.159896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.160007 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.160118 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.160229 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.160342 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.160513 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.160628 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.160740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.160842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.160936 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.161030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.161132 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.161227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.161320 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.161414 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.161508 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.161631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.161742 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.161873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.161976 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.162115 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.162223 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.162354 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.162467 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.162591 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.162694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.162825 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.162926 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.163055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.163184 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.164535 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.164678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.164826 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.165000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.165239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.165355 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.165467 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.165626 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.165626 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.166398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.166518 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.166798 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.166995 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.167729 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.167841 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.167938 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.168033 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.168136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.168238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.168335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.168429 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.168532 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.168629 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.168733 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.external = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.168829 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.168923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.169017 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.169017 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.169193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.169297 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] auth.external = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.169393 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.169515 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.169620 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.password = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.169718 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.169816 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.token = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.169816 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.169993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.170096 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.170192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] auth.password = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.170287 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.170381 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] auth.token = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.170475 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.170570 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.170672 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.170769 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.170864 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.170959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.171053 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.171156 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.171285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.171390 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.171486 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.171582 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.171700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.171815 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.171942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.172056 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] credential.caching = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.172170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.172320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.172436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.172550 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.172671 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.172794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.172910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.173029 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.173029 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.173251 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.173375 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.173491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.173491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.173709 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.173709 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.173923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.174048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.174163 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.174276 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.174398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.174511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.174624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.174745 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.174859 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.174972 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.175085 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.175198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.175342 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.175472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.175587 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.175700 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] federation.caching = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.175813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.175926 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.176039 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.176229 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.176352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.176467 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.176580 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.176693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.176806 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.176927 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.177041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.177155 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.177268 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.177387 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.177509 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.177624 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.177738 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.177859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.177973 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.178087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.178194 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.178308 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.178422 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.178535 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.caching = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.178648 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.178761 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.178881 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.179273 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.179402 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.179543 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.179661 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.179846 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.179978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.180094 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.180207 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.180322 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.180434 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.180547 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.180667 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.180782 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.180895 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.181008 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.181120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.181233 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.181353 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.181468 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.181581 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.181694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.181807 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.181919 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.182040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.182155 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.182268 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.182382 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.182533 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.182781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.182920 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.183037 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.183423 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.183567 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.183685 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.183799 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.183921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.184041 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.184163 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.184278 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.184391 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.184513 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.184635 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.184749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.184863 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.184976 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.185089 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.185202 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.185323 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.185438 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.185552 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.185666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.185779 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.185892 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.186012 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.186127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.186240 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.186353 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.186466 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.186580 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.186705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.186820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.186940 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.187060 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.187160 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.187278 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.187391 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.187508 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.187623 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.187737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.187850 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.187963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.188087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.188202 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.188321 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.188437 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.188550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.188670 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.188795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.188911 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.189024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.189137 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.189250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.189363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.189484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.189605 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.189720 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.189834 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.189947 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.190069 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.190184 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.190298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.190410 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.190523 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.190637 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.190759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.190874 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.190995 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.191110 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.191278 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.191396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.191523 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.191639 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.191752 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.191866 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.191979 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.192092 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.192213 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.192328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.192441 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.192554 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.192668 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.192781 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.192903 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.193018 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.193132 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.193245 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.193358 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.193472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.193592 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.193707 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.193820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.193943 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.194059 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.194059 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.194285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.194407 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.194521 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.194640 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.194755 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.194869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.194991 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.194991 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.195207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.195342 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.195455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.195568 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.195693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.195809 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.195923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.196037 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.196150 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.196150 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.196375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.196490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.196603 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.196716 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.196829 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.196942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.197065 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.197180 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.197293 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.197406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.197519 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.197519 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.197743 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.197743 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.197958 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.198072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.198212 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.198325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.198450 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.198565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.198679 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.198679 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.198892 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.199006 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.199192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.199338 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.199453 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.199566 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.199680 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.199792 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.199919 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.200034 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.200147 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] resource.caching = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.200261 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.200374 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.200487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.200608 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.200723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.200836 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.200949 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.201062 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.201175 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.201296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.201411 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] role.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.201524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.201637 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] role.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.201750 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.201848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.201952 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.202066 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.202179 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.202293 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.202407 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] role.caching = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.202519 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.202641 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] role.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.202755 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.202868 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.202980 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.203094 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.203231 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.203350 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.203464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.203578 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.203690 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.203803 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.203931 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.204047 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.204160 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.204273 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.204387 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.204500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.204621 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.204737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.204850 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.204963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.204963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.205174 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.205295 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.205411 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.205525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.205638 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.205752 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.205865 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.205987 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.206100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.206213 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.206326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.206440 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.206553 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.206674 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.206789 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.206902 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.207015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.207128 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.207285 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.207427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.207594 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.207738 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.207904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.208025 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.208150 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.208265 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.208380 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.208494 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.208609 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.208723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.208845 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.208960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.209074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.209188 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.209302 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.209415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.209536 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.209653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.209766 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.209880 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.209994 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.210107 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.210233 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.210349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.210463 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.210576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.210691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] token.caching = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.210804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.210926 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.211042 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.211146 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.211273 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.211388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.211501 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.211626 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.211742 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.211855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.211968 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.212080 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.212724 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.212849 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.212965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.213078 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.213191 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.213304 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.213418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.213539 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.213654 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.214290 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.214412 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.214563 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.214708 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.214871 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.214992 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.connection = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.215116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.215322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.215480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.215604 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.215720 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.215842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.215960 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.216074 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.216188 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.connection = **** {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.216310 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.216433 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.216548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.216662 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.216783 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.216899 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.217014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.217127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.217241 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.217354 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.217476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.217592 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.217705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.217819 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.217933 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.218048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.218169 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.218285 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.218398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.218511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.218624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.218738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.218927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.218927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.219148 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.219276 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.219392 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.219516 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 07 14:06:26.219640 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.219640 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.219855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.220093 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.220473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:06:26.220647 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 07 14:06:26.249818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.250022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.250110 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.250262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.250355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.250475 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.259735 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.260103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.289232 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.289456 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.289804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.290035 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.344799 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.345261 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.394566 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.394799 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.395773 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.396009 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.462401 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.462598 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.462707 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.462831 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.462915 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.463050 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.463132 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.463289 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.463390 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.463509 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.463592 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.464412 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.488292 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.488548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.488641 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.488807 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.488900 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.489025 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.489119 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.489239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.489321 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.489450 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.489532 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.489662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.489755 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.489868 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.501664 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.501944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.514503 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.514757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.515017 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.515101 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.515264 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.515346 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.515466 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.515549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.515679 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.515760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.515891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.515973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.516116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.516327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:06:26.533971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.534181 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 14:06:26.534442 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.534541 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.534650 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 14:06:26.534731 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 14:06:26.534952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.535032 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.535167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 14:06:26.535275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.535459 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.535539 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.535715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.535794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.535972 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.536225 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.547106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.547403 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.547674 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.547817 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.548091 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.548171 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.548357 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.548794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.556468 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.557188 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.560361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.560638 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.560916 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.561193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.561286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.561473 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.561555 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.561732 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.561813 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.562102 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.562198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.562273 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.562346 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.562520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.562602 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.562789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.562869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.563054 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.563133 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.563407 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.563678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.564037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.589757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.591109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.591388 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.591668 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.592747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.593020 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.597093 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.597330 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.597608 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.598367 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.598643 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.598916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.605585 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.605864 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.607072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.607378 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.610337 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.610620 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.610900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.611182 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.611480 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.611752 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.611839 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.612084 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.612163 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.612360 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.612442 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.612638 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.612909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.613179 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.613552 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.613820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.631919 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.632213 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.633491 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.633773 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.637011 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.638661 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.640676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.640969 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.642313 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.642605 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.645970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.647618 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.649585 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.651188 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.653281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.653576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.653859 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.654140 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.654427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.654824 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.655166 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.655483 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.655840 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.656124 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.656410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.656796 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.674220 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.674503 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.674776 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.675035 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.676506 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.676788 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.677051 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.677307 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.684524 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.686339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.686773 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.688794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.689877 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.690166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.690447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.690588 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.690727 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.690862 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.691018 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.693921 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.694202 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.694481 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.694757 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.695044 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.705143 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.705428 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.705706 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.705983 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.706259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.706532 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.706814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.709292 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.709570 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.709845 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.710119 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.710395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.710664 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.710948 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.724877 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.728293 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.729003 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.732404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.733308 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.734005 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.735682 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.735961 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.736239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.736625 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.738217 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.740941 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.741344 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.741629 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.743601 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.745410 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.748902 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.749189 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.749472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.749758 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.750096 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.751864 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.755041 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.755346 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.755629 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.755912 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.760594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.760888 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.761167 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.765259 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.765543 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.765817 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.769161 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.769437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.769711 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.769982 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.770248 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.770521 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.770800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.771077 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.771371 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.771638 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.771903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.772167 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.772441 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.772700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.773758 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:06:26.774033 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.774306 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.774575 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.774841 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.775107 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.775411 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.775690 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.775955 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.776745 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.777189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.777537 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.777819 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.778079 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:06:26.812156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5636b57729c0 pid: 80233 (default app) Jun 07 14:06:26.815430 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278edd3-30ea-41fe-bf30-815606809b01 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:26.815722 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278edd3-30ea-41fe-bf30-815606809b01 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:26.815999 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278edd3-30ea-41fe-bf30-815606809b01 None None] PATH_INFO: `/v3` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:26.816741 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5636b57729c0 pid: 80234 (default app) Jun 07 14:06:26.820290 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1/1] 10.222.0.81 () {50 vars in 791 bytes} [Tue Jun 7 14:06:26 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:06:34.773416 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fe3fd-5509-44c8-a1f3-168a96413abb None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:34.773984 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fe3fd-5509-44c8-a1f3-168a96413abb None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:34.773984 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fe3fd-5509-44c8-a1f3-168a96413abb None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:34.779366 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1/2] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:06:34 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:06:34.784854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366e4e0-7c8d-4446-8e63-51399edc9e8f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:34.785267 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366e4e0-7c8d-4446-8e63-51399edc9e8f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:34.785421 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366e4e0-7c8d-4446-8e63-51399edc9e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:34.857291 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: INFO dbcounter [None req-f366e4e0-7c8d-4446-8e63-51399edc9e8f None None] Registered counter for database keystone Jun 07 14:06:34.878941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG oslo_db.sqlalchemy.engines [None req-f366e4e0-7c8d-4446-8e63-51399edc9e8f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80233) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 07 14:06:34.886586 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writer thread running {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 07 14:06:35.172668 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG passlib.handlers.bcrypt [None req-f366e4e0-7c8d-4446-8e63-51399edc9e8f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80233) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 07 14:06:35.173102 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG passlib.handlers.bcrypt [None req-f366e4e0-7c8d-4446-8e63-51399edc9e8f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80233) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 07 14:06:35.229147 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f366e4e0-7c8d-4446-8e63-51399edc9e8f None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:06:35.230837 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f366e4e0-7c8d-4446-8e63-51399edc9e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:06:35.353794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 2/3] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:06:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 570 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 07 14:06:35.533134 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44fe3fd-5509-44c8-a1f3-168a96413abb None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:35.603118 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: INFO dbcounter [None req-f44fe3fd-5509-44c8-a1f3-168a96413abb None None] Registered counter for database keystone Jun 07 14:06:35.623917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG oslo_db.sqlalchemy.engines [None req-f44fe3fd-5509-44c8-a1f3-168a96413abb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80234) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 07 14:06:35.630088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writer thread running {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 07 14:06:35.852346 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44fe3fd-5509-44c8-a1f3-168a96413abb None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:35.857327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa445c2-f26b-449d-bd63-f27d71c23539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:35.857988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa445c2-f26b-449d-bd63-f27d71c23539 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:35.858276 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa445c2-f26b-449d-bd63-f27d71c23539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:35.858559 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa445c2-f26b-449d-bd63-f27d71c23539 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:35.866622 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa445c2-f26b-449d-bd63-f27d71c23539 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:35.925661 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0aa445c2-f26b-449d-bd63-f27d71c23539 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 07 14:06:35.926007 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa445c2-f26b-449d-bd63-f27d71c23539 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:35.936047 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 2/4] 10.222.0.81 () {58 vars in 1324 bytes} [Tue Jun 7 14:06:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:06:35.942097 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f366e4e0-7c8d-4446-8e63-51399edc9e8f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:35.956489 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f366e4e0-7c8d-4446-8e63-51399edc9e8f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:35.960934 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7331026-9b6f-428b-b158-b984512575f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:35.962071 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7331026-9b6f-428b-b158-b984512575f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:35.962359 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7331026-9b6f-428b-b158-b984512575f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:35.962636 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7331026-9b6f-428b-b158-b984512575f3 admin admin] PATH_INFO: `/v3/projects/50e31e6b5bc94b188158e931e4ee4076` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:35.964844 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7331026-9b6f-428b-b158-b984512575f3 admin admin] RBAC: Authorizing `identity:get_project(project_id=50e31e6b5bc94b188158e931e4ee4076)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:36.015527 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d7331026-9b6f-428b-b158-b984512575f3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 07 14:06:36.015872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7331026-9b6f-428b-b158-b984512575f3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:36.021756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 3/5] 10.222.0.81 () {56 vars in 1207 bytes} [Tue Jun 7 14:06:35 2022] GET /identity/v3/projects/50e31e6b5bc94b188158e931e4ee4076 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:06:38.252556 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b0a3d-586e-41b7-9a1b-8e64c9f1909f None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:38.253070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b0a3d-586e-41b7-9a1b-8e64c9f1909f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:38.253169 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b0a3d-586e-41b7-9a1b-8e64c9f1909f None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:38.257231 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 3/6] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:06:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:06:38.262775 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102ab25-0a31-4fb6-bf3c-a5813507aeca None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:38.263181 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102ab25-0a31-4fb6-bf3c-a5813507aeca None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:38.263377 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102ab25-0a31-4fb6-bf3c-a5813507aeca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:38.282366 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9102ab25-0a31-4fb6-bf3c-a5813507aeca None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:06:38.283939 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9102ab25-0a31-4fb6-bf3c-a5813507aeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:06:38.292464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 4/7] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:06:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 07 14:06:38.315107 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b0a3d-586e-41b7-9a1b-8e64c9f1909f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:38.327549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b0a3d-586e-41b7-9a1b-8e64c9f1909f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:38.331753 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09f810-f809-4222-8469-bd4184968548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:38.332385 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09f810-f809-4222-8469-bd4184968548 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:38.332670 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09f810-f809-4222-8469-bd4184968548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:38.332948 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09f810-f809-4222-8469-bd4184968548 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:38.341301 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09f810-f809-4222-8469-bd4184968548 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:38.343059 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09f810-f809-4222-8469-bd4184968548 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:38.351749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 4/8] 10.222.0.81 () {58 vars in 1324 bytes} [Tue Jun 7 14:06:38 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:06:38.357620 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102ab25-0a31-4fb6-bf3c-a5813507aeca None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:38.366397 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102ab25-0a31-4fb6-bf3c-a5813507aeca None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:38.370937 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7c685-cb7e-40a5-b4e5-c1b3bdb4ce80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:38.371825 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7c685-cb7e-40a5-b4e5-c1b3bdb4ce80 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:38.372107 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7c685-cb7e-40a5-b4e5-c1b3bdb4ce80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:38.372379 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7c685-cb7e-40a5-b4e5-c1b3bdb4ce80 admin admin] PATH_INFO: `/v3/users/2604a49f226a4bdf958c4047caf193e8` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:38.374032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b7c685-cb7e-40a5-b4e5-c1b3bdb4ce80 admin admin] RBAC: Authorizing `identity:get_user(user_id=2604a49f226a4bdf958c4047caf193e8)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:38.375876 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b7c685-cb7e-40a5-b4e5-c1b3bdb4ce80 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:38.377526 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 5/9] 10.222.0.81 () {56 vars in 1198 bytes} [Tue Jun 7 14:06:38 2022] GET /identity/v3/users/2604a49f226a4bdf958c4047caf193e8 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:06:40.669273 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00b89b-3d13-42f8-8baf-3950153234d3 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:40.669790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00b89b-3d13-42f8-8baf-3950153234d3 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:40.669901 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00b89b-3d13-42f8-8baf-3950153234d3 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:40.671190 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 5/10] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:06:40.676377 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a89a1-90c2-4eb6-96c5-32edb730e224 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:40.676792 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a89a1-90c2-4eb6-96c5-32edb730e224 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:40.676933 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a89a1-90c2-4eb6-96c5-32edb730e224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:40.693899 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-196a89a1-90c2-4eb6-96c5-32edb730e224 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:06:40.695059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-196a89a1-90c2-4eb6-96c5-32edb730e224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:06:40.727270 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55041f6-ab15-46e5-9a7d-ba1d576b89a5 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:40.727577 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55041f6-ab15-46e5-9a7d-ba1d576b89a5 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:40.727861 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55041f6-ab15-46e5-9a7d-ba1d576b89a5 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:40.729824 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 6/11] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:06:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:06:40.735021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc2d07-9a8b-477d-952b-39a707dfe927 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:40.735339 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc2d07-9a8b-477d-952b-39a707dfe927 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:40.735619 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc2d07-9a8b-477d-952b-39a707dfe927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:40.775514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 6/12] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:06:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:06:40.777751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG passlib.handlers.bcrypt [None req-d2fc2d07-9a8b-477d-952b-39a707dfe927 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80234) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 07 14:06:40.778186 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG passlib.handlers.bcrypt [None req-d2fc2d07-9a8b-477d-952b-39a707dfe927 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80234) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 07 14:06:40.798890 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a89a1-90c2-4eb6-96c5-32edb730e224 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:40.821161 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:40.824708 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-d2fc2d07-9a8b-477d-952b-39a707dfe927 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:06:40.826333 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d2fc2d07-9a8b-477d-952b-39a707dfe927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:06:40.902766 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a89a1-90c2-4eb6-96c5-32edb730e224 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:40.925968 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 7/13] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:06:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:06:40.936120 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5f032-1b1f-401c-b8c1-371fb673823f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:40.936925 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5f032-1b1f-401c-b8c1-371fb673823f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:40.937214 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5f032-1b1f-401c-b8c1-371fb673823f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:40.937496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5f032-1b1f-401c-b8c1-371fb673823f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:40.948714 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc2d07-9a8b-477d-952b-39a707dfe927 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:40.949335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5f032-1b1f-401c-b8c1-371fb673823f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:06:40.960439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f3c5f032-1b1f-401c-b8c1-371fb673823f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:06:40.960439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:06:40.960439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:06:40.960439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:06:40.960439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:06:40.960439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:06:40.960439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:06:40.960439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:06:40.960439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:06:40.960439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:06:40.960439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:06:40.960439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:06:40.960439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:06:40.960439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:06:40.960439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:06:40.960439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:06:40.961624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:06:40.961624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:06:40.961624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:06:40.961624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:06:40.961624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:06:40.961624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:06:40.961624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:06:40.961624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:06:40.961624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:06:40.961624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:06:40.961624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:06:40.961624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:06:40.961624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:06:40.961624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:06:40.961624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:06:40.961624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:06:40.962891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:06:40.962891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:06:40.962891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:06:40.962891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:06:40.962891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:06:40.962891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:06:40.962891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:06:40.962891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:06:40.962891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:06:40.962891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:06:40.962891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:06:40.962891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:06:40.962891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 7/14] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:06:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:06:40.968079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5f032-1b1f-401c-b8c1-371fb673823f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:41.048089 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:41.049616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc2d07-9a8b-477d-952b-39a707dfe927 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:41.052369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:41.067182 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5f032-1b1f-401c-b8c1-371fb673823f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:41.087651 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb89899-e0f5-4129-a694-19a56795c2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:41.088752 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb89899-e0f5-4129-a694-19a56795c2c2 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:41.089063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb89899-e0f5-4129-a694-19a56795c2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:41.089327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb89899-e0f5-4129-a694-19a56795c2c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:41.101331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c221ac-a800-4e47-a6b7-d0ede09f15e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:41.101997 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c221ac-a800-4e47-a6b7-d0ede09f15e1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:41.102282 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c221ac-a800-4e47-a6b7-d0ede09f15e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:41.102559 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c221ac-a800-4e47-a6b7-d0ede09f15e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:41.103344 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c221ac-a800-4e47-a6b7-d0ede09f15e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:06:41.103653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c221ac-a800-4e47-a6b7-d0ede09f15e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:41.105160 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c221ac-a800-4e47-a6b7-d0ede09f15e1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:41.116903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 8/15] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:06:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:06:41.123171 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c221ac-a800-4e47-a6b7-d0ede09f15e1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:41.159695 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb89899-e0f5-4129-a694-19a56795c2c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:41.161225 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb89899-e0f5-4129-a694-19a56795c2c2 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:41.225360 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c221ac-a800-4e47-a6b7-d0ede09f15e1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:41.259298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1f899-49e7-4a6e-9bae-2493bdd4f733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:41.260204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1f899-49e7-4a6e-9bae-2493bdd4f733 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:41.260508 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1f899-49e7-4a6e-9bae-2493bdd4f733 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:41.260839 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1f899-49e7-4a6e-9bae-2493bdd4f733 None admin] PATH_INFO: `/v3/users/2604a49f226a4bdf958c4047caf193e8` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:41.260928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 8/16] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:06:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 14:06:41.262068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c1f899-49e7-4a6e-9bae-2493bdd4f733 None admin] RBAC: Authorizing `identity:get_user(user_id=2604a49f226a4bdf958c4047caf193e8)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:41.263679 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c1f899-49e7-4a6e-9bae-2493bdd4f733 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:41.265430 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 9/17] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:06:41 2022] GET /identity/v3/users/2604a49f226a4bdf958c4047caf193e8 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:06:41.270501 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1f899-49e7-4a6e-9bae-2493bdd4f733 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:41.270625 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb89899-e0f5-4129-a694-19a56795c2c2 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:41.273932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:41.293894 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:41.369136 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb89899-e0f5-4129-a694-19a56795c2c2 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:41.379195 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1f899-49e7-4a6e-9bae-2493bdd4f733 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:41.403470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6cc8e-d72d-49ce-9a23-c0ae7bc3ee53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:41.404263 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6cc8e-d72d-49ce-9a23-c0ae7bc3ee53 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:41.404548 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6cc8e-d72d-49ce-9a23-c0ae7bc3ee53 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:41.404825 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6cc8e-d72d-49ce-9a23-c0ae7bc3ee53 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:41.406202 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6cc8e-d72d-49ce-9a23-c0ae7bc3ee53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:41.407704 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6cc8e-d72d-49ce-9a23-c0ae7bc3ee53 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:41.409322 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 9/18] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:06:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:06:41.412549 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba873da-c0c8-422a-82e7-27b24742f69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:41.413337 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba873da-c0c8-422a-82e7-27b24742f69f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:41.413617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba873da-c0c8-422a-82e7-27b24742f69f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:41.413986 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba873da-c0c8-422a-82e7-27b24742f69f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:41.414931 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6cc8e-d72d-49ce-9a23-c0ae7bc3ee53 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:41.424983 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba873da-c0c8-422a-82e7-27b24742f69f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:41.426431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba873da-c0c8-422a-82e7-27b24742f69f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:41.445646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-fba873da-c0c8-422a-82e7-27b24742f69f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:06:41.445646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:06:41.445646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:06:41.445646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:06:41.445646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:06:41.445646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:06:41.445646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:06:41.445646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:06:41.445646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:06:41.445646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:06:41.445646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:06:41.445646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:06:41.445646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:06:41.445646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:06:41.445646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:06:41.446518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:06:41.446518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:06:41.446518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:06:41.446518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:06:41.446518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:06:41.446518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:06:41.446518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:06:41.446518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:06:41.446518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:06:41.446518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:06:41.446518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:06:41.446518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:06:41.446518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:06:41.446518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:06:41.446518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:06:41.446518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:06:41.446518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:06:41.447363 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:06:41.447363 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:06:41.447363 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:06:41.447363 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:06:41.447363 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 10/19] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:06:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:06:41.453458 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba873da-c0c8-422a-82e7-27b24742f69f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:41.489452 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:41.515722 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:41.520133 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6cc8e-d72d-49ce-9a23-c0ae7bc3ee53 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:41.553636 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03296661-041a-4e31-bb70-f9953bf33708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:41.553833 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba873da-c0c8-422a-82e7-27b24742f69f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:41.554293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03296661-041a-4e31-bb70-f9953bf33708 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:41.554577 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03296661-041a-4e31-bb70-f9953bf33708 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:41.554855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03296661-041a-4e31-bb70-f9953bf33708 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:41.555729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03296661-041a-4e31-bb70-f9953bf33708 None admin] RBAC: Adding query filter params (role.id=82bbe3651b9b4f378fd150d93357b2b1, scope.domain.id=default, user.id=2604a49f226a4bdf958c4047caf193e8) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:06:41.556040 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03296661-041a-4e31-bb70-f9953bf33708 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:41.558095 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03296661-041a-4e31-bb70-f9953bf33708 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:41.567024 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 10/20] 10.222.0.81 () {56 vars in 1322 bytes} [Tue Jun 7 14:06:41 2022] GET /identity/v3/role_assignments?user.id=2604a49f226a4bdf958c4047caf193e8&scope.domain.id=default&role.id=82bbe3651b9b4f378fd150d93357b2b1 => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:06:41.588213 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3d6cb-0065-4204-afcd-866444480b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:41.589034 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3d6cb-0065-4204-afcd-866444480b33 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:41.589368 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3d6cb-0065-4204-afcd-866444480b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:41.589650 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3d6cb-0065-4204-afcd-866444480b33 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:41.590409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3d6cb-0065-4204-afcd-866444480b33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:06:41.590718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3d6cb-0065-4204-afcd-866444480b33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:41.592198 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3d6cb-0065-4204-afcd-866444480b33 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:41.612293 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 11/21] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:06:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:06:44.267333 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf39a74-2d4d-44ba-a1fa-63ee9f26e8c1 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:44.268360 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf39a74-2d4d-44ba-a1fa-63ee9f26e8c1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:44.268852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf39a74-2d4d-44ba-a1fa-63ee9f26e8c1 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:44.270520 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 11/22] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:06:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:06:44.277438 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d1672-ed3d-4b4b-8a59-beed1b3fd55c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:44.277977 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d1672-ed3d-4b4b-8a59-beed1b3fd55c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:44.277977 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d1672-ed3d-4b4b-8a59-beed1b3fd55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:44.304636 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-380d1672-ed3d-4b4b-8a59-beed1b3fd55c None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:06:44.305838 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-380d1672-ed3d-4b4b-8a59-beed1b3fd55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:06:44.395638 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 12/23] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:06:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:06:44.430790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf39a74-2d4d-44ba-a1fa-63ee9f26e8c1 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:44.592930 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf39a74-2d4d-44ba-a1fa-63ee9f26e8c1 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:44.599355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72c7b6-ad4e-4847-91a3-419a2fe899f3 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:44.599923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72c7b6-ad4e-4847-91a3-419a2fe899f3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:44.600406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72c7b6-ad4e-4847-91a3-419a2fe899f3 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:44.601985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 13/24] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:06:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:06:44.607920 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbcaca7-c86a-4d80-a22b-689baaa6cd2a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:44.608447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbcaca7-c86a-4d80-a22b-689baaa6cd2a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:44.608947 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbcaca7-c86a-4d80-a22b-689baaa6cd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:44.651249 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0606bb6-3025-4e07-8abd-0f8f4fba5e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:44.651249 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0606bb6-3025-4e07-8abd-0f8f4fba5e1b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:44.651249 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0606bb6-3025-4e07-8abd-0f8f4fba5e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:44.651249 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0606bb6-3025-4e07-8abd-0f8f4fba5e1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:44.652868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0cbcaca7-c86a-4d80-a22b-689baaa6cd2a None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:06:44.654285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0cbcaca7-c86a-4d80-a22b-689baaa6cd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:06:44.659245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:44.681583 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0606bb6-3025-4e07-8abd-0f8f4fba5e1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:44.683636 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0606bb6-3025-4e07-8abd-0f8f4fba5e1b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:44.722348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-a0606bb6-3025-4e07-8abd-0f8f4fba5e1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:06:44.722348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:06:44.722348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:06:44.722348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:06:44.722348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:06:44.722348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:06:44.722348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:06:44.722348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:06:44.722348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:06:44.722348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:06:44.722348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:06:44.722348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:06:44.722348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:06:44.722348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:06:44.722348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:06:44.723437 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:06:44.723437 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:06:44.723437 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:06:44.723437 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:06:44.723437 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:06:44.723437 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:06:44.723437 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:06:44.723437 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:06:44.723437 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:06:44.723437 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:06:44.723437 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:06:44.723437 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:06:44.723437 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:06:44.723437 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:06:44.723437 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:06:44.723437 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:06:44.723437 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:06:44.724308 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:06:44.724308 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:06:44.724308 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:06:44.724308 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:06:44.724308 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 12/25] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:06:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:06:44.725399 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c823255-df72-48bf-ae9b-adea8971c8bd None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:44.725695 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c823255-df72-48bf-ae9b-adea8971c8bd None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:44.725976 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c823255-df72-48bf-ae9b-adea8971c8bd None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:44.727254 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 13/26] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:06:44.729394 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282db32-bd7e-47c5-9f53-88104eb03f18 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:44.729830 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282db32-bd7e-47c5-9f53-88104eb03f18 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:44.730261 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282db32-bd7e-47c5-9f53-88104eb03f18 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:44.731514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 14/27] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:06:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:06:44.736472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbeebf1-e325-46cd-b6f4-9162972b3326 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:44.736760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbeebf1-e325-46cd-b6f4-9162972b3326 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:44.737036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbeebf1-e325-46cd-b6f4-9162972b3326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:44.750934 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:44.761844 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-0fbeebf1-e325-46cd-b6f4-9162972b3326 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:06:44.764214 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0fbeebf1-e325-46cd-b6f4-9162972b3326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:06:44.769706 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 14/28] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:06:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 07 14:06:44.779238 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbcaca7-c86a-4d80-a22b-689baaa6cd2a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:44.890362 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 15/29] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:06:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:06:44.891476 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56483138-45a4-41a7-8d6d-aae2c56191ca None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:44.891908 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56483138-45a4-41a7-8d6d-aae2c56191ca None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:44.892326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56483138-45a4-41a7-8d6d-aae2c56191ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:44.924723 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-56483138-45a4-41a7-8d6d-aae2c56191ca None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:06:44.928005 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-56483138-45a4-41a7-8d6d-aae2c56191ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:06:44.975911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbcaca7-c86a-4d80-a22b-689baaa6cd2a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:45.047857 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ae711-1cb0-4506-8f4d-156e0ecbf582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:45.048535 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ae711-1cb0-4506-8f4d-156e0ecbf582 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:45.048821 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ae711-1cb0-4506-8f4d-156e0ecbf582 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:45.049098 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ae711-1cb0-4506-8f4d-156e0ecbf582 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:45.049832 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ae711-1cb0-4506-8f4d-156e0ecbf582 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:06:45.050140 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ae711-1cb0-4506-8f4d-156e0ecbf582 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:45.051537 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ae711-1cb0-4506-8f4d-156e0ecbf582 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:45.067698 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 16/30] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:06:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:06:45.068746 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56483138-45a4-41a7-8d6d-aae2c56191ca None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:45.078771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 15/31] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:06:45.080631 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ae711-1cb0-4506-8f4d-156e0ecbf582 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:45.147362 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:45.163926 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:45.240933 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ae711-1cb0-4506-8f4d-156e0ecbf582 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:45.268027 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56483138-45a4-41a7-8d6d-aae2c56191ca None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:45.310813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece774a-4382-4acb-8528-eb42d3a992c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:45.311730 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece774a-4382-4acb-8528-eb42d3a992c6 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:45.312136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece774a-4382-4acb-8528-eb42d3a992c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:45.312604 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece774a-4382-4acb-8528-eb42d3a992c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:45.313461 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece774a-4382-4acb-8528-eb42d3a992c6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:45.315121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece774a-4382-4acb-8528-eb42d3a992c6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:45.324583 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865bf84c-bfc2-487c-85aa-a62615c3891a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:45.326287 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865bf84c-bfc2-487c-85aa-a62615c3891a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:45.326804 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865bf84c-bfc2-487c-85aa-a62615c3891a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:45.327441 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865bf84c-bfc2-487c-85aa-a62615c3891a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:45.343121 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865bf84c-bfc2-487c-85aa-a62615c3891a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:06:45.351959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-865bf84c-bfc2-487c-85aa-a62615c3891a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:06:45.351959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:06:45.351959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:06:45.351959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:06:45.351959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:06:45.351959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:06:45.351959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:06:45.351959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:06:45.351959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:06:45.351959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:06:45.351959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:06:45.351959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:06:45.351959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:06:45.351959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:06:45.351959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:06:45.351959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:06:45.353021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:06:45.353021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:06:45.353021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:06:45.353021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:06:45.353021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:06:45.353021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:06:45.353021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:06:45.353021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:06:45.353021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:06:45.353021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:06:45.353021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:06:45.353021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:06:45.353021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:06:45.353021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:06:45.353021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:06:45.353021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:06:45.354274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:06:45.354274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:06:45.354274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:06:45.354274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:06:45.354274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:06:45.354274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:06:45.354274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:06:45.354274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:06:45.354274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:06:45.354274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:06:45.354274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:06:45.354274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:06:45.354274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 17/32] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:06:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:06:45.354274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865bf84c-bfc2-487c-85aa-a62615c3891a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:45.451641 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 16/33] 10.222.0.81 () {60 vars in 1131 bytes} [Tue Jun 7 14:06:45 2022] POST /identity/v3/roles => generated 219 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:06:45.460534 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece774a-4382-4acb-8528-eb42d3a992c6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:45.497496 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865bf84c-bfc2-487c-85aa-a62615c3891a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:45.517222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:45.576068 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e3df7-3439-4a90-9806-7a990835e907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:45.576774 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e3df7-3439-4a90-9806-7a990835e907 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:45.577064 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e3df7-3439-4a90-9806-7a990835e907 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:45.577343 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e3df7-3439-4a90-9806-7a990835e907 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:45.578085 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e3df7-3439-4a90-9806-7a990835e907 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:45.579662 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e3df7-3439-4a90-9806-7a990835e907 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:45.583719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece774a-4382-4acb-8528-eb42d3a992c6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:45.595698 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:45.631802 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79718a1b-fe60-405a-95a1-9863dcd89389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:45.632509 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79718a1b-fe60-405a-95a1-9863dcd89389 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:45.633000 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79718a1b-fe60-405a-95a1-9863dcd89389 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:45.634608 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79718a1b-fe60-405a-95a1-9863dcd89389 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:45.635824 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79718a1b-fe60-405a-95a1-9863dcd89389 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:45.638495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79718a1b-fe60-405a-95a1-9863dcd89389 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:45.668285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 17/34] 10.222.0.81 () {60 vars in 1131 bytes} [Tue Jun 7 14:06:45 2022] POST /identity/v3/roles => generated 225 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:06:45.669206 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79718a1b-fe60-405a-95a1-9863dcd89389 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:45.825955 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 18/35] 10.222.0.81 () {60 vars in 1141 bytes} [Tue Jun 7 14:06:45 2022] POST /identity/v3/projects => generated 301 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:06:45.827608 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e51365-1123-4ced-a4f8-eba7f2c54f72 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:45.827907 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e51365-1123-4ced-a4f8-eba7f2c54f72 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:45.828191 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e51365-1123-4ced-a4f8-eba7f2c54f72 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:45.828682 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79718a1b-fe60-405a-95a1-9863dcd89389 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:45.831130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 19/36] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:06:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:06:45.838342 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ba31f-7bf1-45ac-8fee-46a10df80714 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:45.838636 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ba31f-7bf1-45ac-8fee-46a10df80714 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:45.839108 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ba31f-7bf1-45ac-8fee-46a10df80714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:45.870975 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-dc1ba31f-7bf1-45ac-8fee-46a10df80714 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:06:45.873030 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-dc1ba31f-7bf1-45ac-8fee-46a10df80714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:06:45.882025 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4420746-ff1e-4fd0-91f2-dd0efd3ace33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:45.883086 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4420746-ff1e-4fd0-91f2-dd0efd3ace33 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:45.883618 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4420746-ff1e-4fd0-91f2-dd0efd3ace33 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:45.884076 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4420746-ff1e-4fd0-91f2-dd0efd3ace33 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:45.885910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4420746-ff1e-4fd0-91f2-dd0efd3ace33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:06:45.886542 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4420746-ff1e-4fd0-91f2-dd0efd3ace33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:45.888189 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4420746-ff1e-4fd0-91f2-dd0efd3ace33 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:45.898179 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 18/37] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:06:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:06:45.904305 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4420746-ff1e-4fd0-91f2-dd0efd3ace33 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:45.939667 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:46.005415 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 20/38] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:06:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:06:46.029735 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4420746-ff1e-4fd0-91f2-dd0efd3ace33 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:46.037248 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ba31f-7bf1-45ac-8fee-46a10df80714 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:46.071110 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d8e4c-b369-4635-8ade-3b13fe87c9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:46.072457 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d8e4c-b369-4635-8ade-3b13fe87c9fc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:46.072924 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d8e4c-b369-4635-8ade-3b13fe87c9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:46.073485 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d8e4c-b369-4635-8ade-3b13fe87c9fc None admin] PATH_INFO: `/v3/users/2604a49f226a4bdf958c4047caf193e8` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:46.075236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d8e4c-b369-4635-8ade-3b13fe87c9fc None admin] RBAC: Authorizing `identity:get_user(user_id=2604a49f226a4bdf958c4047caf193e8)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:46.077188 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d8e4c-b369-4635-8ade-3b13fe87c9fc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:46.079265 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 19/39] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:06:45 2022] GET /identity/v3/users/2604a49f226a4bdf958c4047caf193e8 => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:06:46.085569 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d8e4c-b369-4635-8ade-3b13fe87c9fc None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:46.158448 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:46.174002 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ba31f-7bf1-45ac-8fee-46a10df80714 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:46.213024 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5128635-e6c4-4886-9ffe-d3d984f438b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:46.214464 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5128635-e6c4-4886-9ffe-d3d984f438b3 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:46.214818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5128635-e6c4-4886-9ffe-d3d984f438b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:46.215133 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5128635-e6c4-4886-9ffe-d3d984f438b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:46.215664 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:46.215902 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5128635-e6c4-4886-9ffe-d3d984f438b3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:46.217563 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5128635-e6c4-4886-9ffe-d3d984f438b3 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:46.225882 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d8e4c-b369-4635-8ade-3b13fe87c9fc None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:46.240474 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 21/40] 10.222.0.81 () {60 vars in 1131 bytes} [Tue Jun 7 14:06:46 2022] POST /identity/v3/roles => generated 223 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:06:46.242383 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ed755-10d6-4d67-a4eb-a8f55dd97245 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:46.243028 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ed755-10d6-4d67-a4eb-a8f55dd97245 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:46.243425 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ed755-10d6-4d67-a4eb-a8f55dd97245 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:46.246075 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 22/41] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:06:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:06:46.251922 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f71c7-8cc6-4571-823a-211f88ad04b6 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:46.252211 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f71c7-8cc6-4571-823a-211f88ad04b6 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:46.254542 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f71c7-8cc6-4571-823a-211f88ad04b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:46.272277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-399f71c7-8cc6-4571-823a-211f88ad04b6 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:06:46.273569 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-399f71c7-8cc6-4571-823a-211f88ad04b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:06:46.276313 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120ffc8-7f8e-43e4-ae07-92a321612a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:46.277189 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120ffc8-7f8e-43e4-ae07-92a321612a9c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:46.277632 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120ffc8-7f8e-43e4-ae07-92a321612a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:46.277930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120ffc8-7f8e-43e4-ae07-92a321612a9c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:46.279519 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0120ffc8-7f8e-43e4-ae07-92a321612a9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:46.281012 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0120ffc8-7f8e-43e4-ae07-92a321612a9c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:46.283028 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 20/42] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:06:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:06:46.288599 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120ffc8-7f8e-43e4-ae07-92a321612a9c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:46.355928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 23/43] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:06:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:06:46.379533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f71c7-8cc6-4571-823a-211f88ad04b6 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:46.405250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120ffc8-7f8e-43e4-ae07-92a321612a9c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:46.441440 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a704f4-2917-4b2f-b5bc-d3bb3bc766d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:46.442331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a704f4-2917-4b2f-b5bc-d3bb3bc766d3 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:46.442646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a704f4-2917-4b2f-b5bc-d3bb3bc766d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:46.442908 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a704f4-2917-4b2f-b5bc-d3bb3bc766d3 None admin] PATH_INFO: `/v3/domains/default/users/2604a49f226a4bdf958c4047caf193e8/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:46.445335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a704f4-2917-4b2f-b5bc-d3bb3bc766d3 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=2604a49f226a4bdf958c4047caf193e8, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:46.446444 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a704f4-2917-4b2f-b5bc-d3bb3bc766d3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:46.463522 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 21/44] 10.222.0.81 () {58 vars in 1361 bytes} [Tue Jun 7 14:06:46 2022] PUT /identity/v3/domains/default/users/2604a49f226a4bdf958c4047caf193e8/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:06:46.479051 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:46.504922 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f71c7-8cc6-4571-823a-211f88ad04b6 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:46.542699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f5441-eb33-4db1-84be-cd0d38acc11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:46.543563 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f5441-eb33-4db1-84be-cd0d38acc11f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:46.543850 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f5441-eb33-4db1-84be-cd0d38acc11f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:46.544127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f5441-eb33-4db1-84be-cd0d38acc11f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:46.545331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f5441-eb33-4db1-84be-cd0d38acc11f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:46.546832 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f5441-eb33-4db1-84be-cd0d38acc11f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:46.548583 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 24/45] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:06:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:06:46.554322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a704f4-2917-4b2f-b5bc-d3bb3bc766d3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:46.582198 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:46.664949 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a704f4-2917-4b2f-b5bc-d3bb3bc766d3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:46.698929 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f9093-8978-4bf6-b43e-810d4b8ac736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:46.699629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f9093-8978-4bf6-b43e-810d4b8ac736 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:46.699921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f9093-8978-4bf6-b43e-810d4b8ac736 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:46.700205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f9093-8978-4bf6-b43e-810d4b8ac736 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:46.700928 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f9093-8978-4bf6-b43e-810d4b8ac736 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:46.702397 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f9093-8978-4bf6-b43e-810d4b8ac736 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:46.728341 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 22/46] 10.222.0.81 () {60 vars in 1141 bytes} [Tue Jun 7 14:06:46 2022] POST /identity/v3/projects => generated 312 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:06:48.721647 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a743a-273c-4917-9460-4a25b5835415 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:48.722165 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a743a-273c-4917-9460-4a25b5835415 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:48.722277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a743a-273c-4917-9460-4a25b5835415 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:48.723616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 25/47] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:06:48.729220 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238327ef-067d-43a1-a9e3-225fc494f2e3 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:48.729638 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238327ef-067d-43a1-a9e3-225fc494f2e3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:48.729777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238327ef-067d-43a1-a9e3-225fc494f2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:48.747150 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-238327ef-067d-43a1-a9e3-225fc494f2e3 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:06:48.748928 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-238327ef-067d-43a1-a9e3-225fc494f2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:06:48.814511 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 23/48] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:06:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:06:48.838330 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a743a-273c-4917-9460-4a25b5835415 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:48.943434 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a743a-273c-4917-9460-4a25b5835415 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:48.978669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3612e-9bde-4e67-a070-610c9cf53c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:48.979501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3612e-9bde-4e67-a070-610c9cf53c1b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:48.979787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3612e-9bde-4e67-a070-610c9cf53c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:48.979965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c24aec-8cc9-4669-a5fa-b1f3539109c5 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:48.980057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3612e-9bde-4e67-a070-610c9cf53c1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:48.980262 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c24aec-8cc9-4669-a5fa-b1f3539109c5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:48.980544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c24aec-8cc9-4669-a5fa-b1f3539109c5 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:48.981930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 24/49] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:06:48.986937 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cb0af-9bbc-4b62-a024-3d414c8a0967 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:48.987252 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cb0af-9bbc-4b62-a024-3d414c8a0967 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:48.987538 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cb0af-9bbc-4b62-a024-3d414c8a0967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:48.990389 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e3612e-9bde-4e67-a070-610c9cf53c1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:06:48.993340 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:48.998764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-a5e3612e-9bde-4e67-a070-610c9cf53c1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:06:48.998764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:06:48.998764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:06:48.998764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:06:48.998764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:06:48.998764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:06:48.998764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:06:48.998764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:06:48.998764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:06:48.998764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:06:48.998764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:06:48.998764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:06:48.998764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:06:48.998764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:06:48.998764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:06:48.998764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:06:48.999699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:06:48.999699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:06:48.999699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:06:48.999699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:06:48.999699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:06:48.999699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:06:48.999699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:06:48.999699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:06:48.999699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:06:48.999699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:06:48.999699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:06:48.999699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:06:48.999699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:06:48.999699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:06:48.999699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:06:48.999699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:06:49.000477 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:06:49.000477 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:06:49.000477 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:06:49.000477 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:06:49.000477 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:06:49.000477 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:06:49.000477 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:06:49.000477 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:06:49.000477 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:06:49.000477 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:06:49.000477 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:06:49.000477 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:06:49.000477 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 26/50] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:06:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:06:49.005160 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-aa8cb0af-9bbc-4b62-a024-3d414c8a0967 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:06:49.005754 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3612e-9bde-4e67-a070-610c9cf53c1b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:49.006518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-aa8cb0af-9bbc-4b62-a024-3d414c8a0967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:06:49.017297 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:49.074671 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 25/51] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:06:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:06:49.098048 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cb0af-9bbc-4b62-a024-3d414c8a0967 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:49.106938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3612e-9bde-4e67-a070-610c9cf53c1b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:49.141667 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3c039-0812-4460-91e8-3f00bdb0ba1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:49.142329 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3c039-0812-4460-91e8-3f00bdb0ba1b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:49.142614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3c039-0812-4460-91e8-3f00bdb0ba1b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:49.142890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3c039-0812-4460-91e8-3f00bdb0ba1b None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:49.143671 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3c039-0812-4460-91e8-3f00bdb0ba1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:06:49.143982 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3c039-0812-4460-91e8-3f00bdb0ba1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:49.145354 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3c039-0812-4460-91e8-3f00bdb0ba1b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:49.157485 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 27/52] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:06:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:06:49.163897 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3c039-0812-4460-91e8-3f00bdb0ba1b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:49.196615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cb0af-9bbc-4b62-a024-3d414c8a0967 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:49.231203 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954eeb4-73f9-45ee-93a6-ba0a747144b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:49.231958 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954eeb4-73f9-45ee-93a6-ba0a747144b7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:49.232250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954eeb4-73f9-45ee-93a6-ba0a747144b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:49.232530 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954eeb4-73f9-45ee-93a6-ba0a747144b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:49.233159 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:49.233725 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954eeb4-73f9-45ee-93a6-ba0a747144b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:49.235190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954eeb4-73f9-45ee-93a6-ba0a747144b7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:49.237216 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 26/53] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:06:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:06:49.242616 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954eeb4-73f9-45ee-93a6-ba0a747144b7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:49.246461 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:49.270804 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3c039-0812-4460-91e8-3f00bdb0ba1b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:49.304739 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b2ab8-e8e3-4649-9b0f-8bef29e86b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:49.305645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b2ab8-e8e3-4649-9b0f-8bef29e86b31 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:49.305935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b2ab8-e8e3-4649-9b0f-8bef29e86b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:49.306209 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b2ab8-e8e3-4649-9b0f-8bef29e86b31 None admin] PATH_INFO: `/v3/users/2604a49f226a4bdf958c4047caf193e8` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:49.307752 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b2ab8-e8e3-4649-9b0f-8bef29e86b31 None admin] RBAC: Authorizing `identity:get_user(user_id=2604a49f226a4bdf958c4047caf193e8)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:49.309225 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b2ab8-e8e3-4649-9b0f-8bef29e86b31 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:49.310875 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 28/54] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:06:49 2022] GET /identity/v3/users/2604a49f226a4bdf958c4047caf193e8 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:06:49.316218 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b2ab8-e8e3-4649-9b0f-8bef29e86b31 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:49.343495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954eeb4-73f9-45ee-93a6-ba0a747144b7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:49.377657 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b285a4-f52e-4b15-80a3-7e49bfc9d952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:49.378325 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b285a4-f52e-4b15-80a3-7e49bfc9d952 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:49.378613 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b285a4-f52e-4b15-80a3-7e49bfc9d952 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:49.378897 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b285a4-f52e-4b15-80a3-7e49bfc9d952 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:49.379730 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b285a4-f52e-4b15-80a3-7e49bfc9d952 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:49.381210 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b285a4-f52e-4b15-80a3-7e49bfc9d952 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:49.407791 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 27/55] 10.222.0.81 () {60 vars in 1140 bytes} [Tue Jun 7 14:06:49 2022] POST /identity/v3/projects => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:06:49.415540 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b2ab8-e8e3-4649-9b0f-8bef29e86b31 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:49.452353 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c92cf-0899-4cf3-a875-912afc9f6a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:49.453206 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c92cf-0899-4cf3-a875-912afc9f6a7e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:49.453495 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c92cf-0899-4cf3-a875-912afc9f6a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:49.453775 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c92cf-0899-4cf3-a875-912afc9f6a7e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:49.455041 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161c92cf-0899-4cf3-a875-912afc9f6a7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:49.456547 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161c92cf-0899-4cf3-a875-912afc9f6a7e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:49.458240 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 29/56] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:06:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:06:49.464977 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b285a4-f52e-4b15-80a3-7e49bfc9d952 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:49.524661 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:49.573782 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b285a4-f52e-4b15-80a3-7e49bfc9d952 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:49.609035 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217fe95-ff72-4cc8-8d4c-5e1849bd52f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:49.609707 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217fe95-ff72-4cc8-8d4c-5e1849bd52f3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:49.609995 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217fe95-ff72-4cc8-8d4c-5e1849bd52f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:49.610296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217fe95-ff72-4cc8-8d4c-5e1849bd52f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:49.611124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217fe95-ff72-4cc8-8d4c-5e1849bd52f3 None admin] RBAC: Adding query filter params (role.id=82bbe3651b9b4f378fd150d93357b2b1, scope.domain.id=default, user.id=2604a49f226a4bdf958c4047caf193e8) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:06:49.611463 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217fe95-ff72-4cc8-8d4c-5e1849bd52f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:49.612917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217fe95-ff72-4cc8-8d4c-5e1849bd52f3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:49.620598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 28/57] 10.222.0.81 () {56 vars in 1322 bytes} [Tue Jun 7 14:06:49 2022] GET /identity/v3/role_assignments?user.id=2604a49f226a4bdf958c4047caf193e8&scope.domain.id=default&role.id=82bbe3651b9b4f378fd150d93357b2b1 => generated 530 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:06:51.667105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39eae6c-0f60-40e5-a9a4-c3cb29f5fd86 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:51.667686 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39eae6c-0f60-40e5-a9a4-c3cb29f5fd86 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:51.667686 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39eae6c-0f60-40e5-a9a4-c3cb29f5fd86 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:51.669059 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 30/58] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:06:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:06:51.674303 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84e7fe-4b6d-443a-8ab4-95b74d73973e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:51.674708 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84e7fe-4b6d-443a-8ab4-95b74d73973e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:51.674876 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84e7fe-4b6d-443a-8ab4-95b74d73973e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:51.692063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ec84e7fe-4b6d-443a-8ab4-95b74d73973e None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:06:51.693253 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ec84e7fe-4b6d-443a-8ab4-95b74d73973e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:06:51.756934 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 29/59] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:06:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:06:51.781629 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39eae6c-0f60-40e5-a9a4-c3cb29f5fd86 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:51.790866 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:51.883289 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39eae6c-0f60-40e5-a9a4-c3cb29f5fd86 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:51.916642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29f3cd-2fa7-4b51-ba1b-71b6f6c8b36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:51.917511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29f3cd-2fa7-4b51-ba1b-71b6f6c8b36f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:51.917707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29f3cd-2fa7-4b51-ba1b-71b6f6c8b36f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:51.917985 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29f3cd-2fa7-4b51-ba1b-71b6f6c8b36f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:51.919160 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29f3cd-2fa7-4b51-ba1b-71b6f6c8b36f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:51.921176 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29f3cd-2fa7-4b51-ba1b-71b6f6c8b36f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:51.922782 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 31/60] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:06:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:06:51.928290 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84e7fe-4b6d-443a-8ab4-95b74d73973e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:51.974843 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:52.023420 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84e7fe-4b6d-443a-8ab4-95b74d73973e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:52.055956 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840ec1c-c9fc-42f4-b75b-0baa7732c93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:52.057282 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840ec1c-c9fc-42f4-b75b-0baa7732c93c None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:52.057555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840ec1c-c9fc-42f4-b75b-0baa7732c93c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:52.057835 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840ec1c-c9fc-42f4-b75b-0baa7732c93c None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:52.058556 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840ec1c-c9fc-42f4-b75b-0baa7732c93c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:52.060040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840ec1c-c9fc-42f4-b75b-0baa7732c93c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:52.084831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 30/61] 10.222.0.81 () {60 vars in 1132 bytes} [Tue Jun 7 14:06:51 2022] POST /identity/v3/users => generated 275 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:06:55.628872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd0f9a-1ec0-4cf5-98c3-683aa245d354 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:55.629771 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd0f9a-1ec0-4cf5-98c3-683aa245d354 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:55.630226 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd0f9a-1ec0-4cf5-98c3-683aa245d354 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:55.631797 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 32/62] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:06:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:06:55.637803 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1ed59-062e-4c73-aab0-3205bf5247a7 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:55.638095 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1ed59-062e-4c73-aab0-3205bf5247a7 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:55.638380 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1ed59-062e-4c73-aab0-3205bf5247a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:55.640007 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132970d9-166d-4dca-995c-9fd695e48616 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:55.640820 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132970d9-166d-4dca-995c-9fd695e48616 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:55.641316 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132970d9-166d-4dca-995c-9fd695e48616 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:55.643188 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 31/63] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:06:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:06:55.648788 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fe729-fdec-4905-960e-fdf7b40fae51 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:55.649354 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fe729-fdec-4905-960e-fdf7b40fae51 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:55.649902 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fe729-fdec-4905-960e-fdf7b40fae51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:55.668295 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-39a1ed59-062e-4c73-aab0-3205bf5247a7 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:06:55.669507 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d85fe729-fdec-4905-960e-fdf7b40fae51 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:06:55.669638 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-39a1ed59-062e-4c73-aab0-3205bf5247a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:06:55.671003 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d85fe729-fdec-4905-960e-fdf7b40fae51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:06:55.751308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 32/64] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:06:55.774351 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fe729-fdec-4905-960e-fdf7b40fae51 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:55.804219 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 33/65] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:06:55.811520 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b217d-963f-4d7b-84f0-ac52c0631b9a None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:55.811938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b217d-963f-4d7b-84f0-ac52c0631b9a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:55.812227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b217d-963f-4d7b-84f0-ac52c0631b9a None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:55.813690 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 34/66] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:06:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:06:55.816632 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:55.828512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fa817-9bf1-4132-baba-f029bf9b5437 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:55.828795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fa817-9bf1-4132-baba-f029bf9b5437 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:55.829230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fa817-9bf1-4132-baba-f029bf9b5437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:55.859819 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-b64fa817-9bf1-4132-baba-f029bf9b5437 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:06:55.861455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b64fa817-9bf1-4132-baba-f029bf9b5437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:06:55.863937 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:55.903966 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fe729-fdec-4905-960e-fdf7b40fae51 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:55.940184 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 35/67] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:06:55.941047 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64fa817-9bf1-4132-baba-f029bf9b5437 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:55.954650 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4595f-0864-41d4-a30c-c53e1838c718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:55.955936 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4595f-0864-41d4-a30c-c53e1838c718 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:55.956769 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4595f-0864-41d4-a30c-c53e1838c718 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:55.957520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4595f-0864-41d4-a30c-c53e1838c718 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:55.970983 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4595f-0864-41d4-a30c-c53e1838c718 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:06:55.978871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-afb4595f-0864-41d4-a30c-c53e1838c718 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:06:55.978871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:06:55.978871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:06:55.978871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:06:55.978871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:06:55.978871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:06:55.978871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:06:55.978871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:06:55.978871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:06:55.978871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:06:55.978871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:06:55.978871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:06:55.978871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:06:55.978871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:06:55.978871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:06:55.978871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:06:55.979848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:06:55.979848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:06:55.979848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:06:55.979848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:06:55.979848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:06:55.979848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:06:55.979848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:06:55.979848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:06:55.979848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:06:55.979848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:06:55.979848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:06:55.979848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:06:55.979848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:06:55.979848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:06:55.979848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:06:55.979848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:06:55.980619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:06:55.980619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:06:55.980619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:06:55.980619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:06:55.980619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:06:55.980619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:06:55.980619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:06:55.980619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:06:55.980619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:06:55.980619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:06:55.980619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:06:55.980619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:06:55.980619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 33/68] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:06:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:06:55.984650 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70814fd-63c6-4f7a-a796-29e2164c9aef None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:55.985555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70814fd-63c6-4f7a-a796-29e2164c9aef None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:55.985915 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70814fd-63c6-4f7a-a796-29e2164c9aef None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:55.987674 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 34/69] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:06:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:06:55.988916 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70814fd-63c6-4f7a-a796-29e2164c9aef None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:56.047747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64fa817-9bf1-4132-baba-f029bf9b5437 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:56.083201 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a4e3a-2f07-44b2-b467-cf8bf8bee1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:56.084035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a4e3a-2f07-44b2-b467-cf8bf8bee1c5 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:56.084314 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a4e3a-2f07-44b2-b467-cf8bf8bee1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:56.084589 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a4e3a-2f07-44b2-b467-cf8bf8bee1c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:56.086748 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:56.090661 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:56.097598 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69a4e3a-2f07-44b2-b467-cf8bf8bee1c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:06:56.098588 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70814fd-63c6-4f7a-a796-29e2164c9aef None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:56.105568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-f69a4e3a-2f07-44b2-b467-cf8bf8bee1c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:06:56.105568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:06:56.105568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:06:56.105568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:06:56.105568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:06:56.105568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:06:56.105568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:06:56.105568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:06:56.105568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:06:56.105568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:06:56.105568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:06:56.105568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:06:56.105568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:06:56.105568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:06:56.105568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:06:56.105568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:06:56.106426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:06:56.106426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:06:56.106426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:06:56.106426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:06:56.106426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:06:56.106426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:06:56.106426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:06:56.106426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:06:56.106426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:06:56.106426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:06:56.106426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:06:56.106426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:06:56.106426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:06:56.106426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:06:56.106426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:06:56.106426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:06:56.108128 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:06:56.108128 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:06:56.108128 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:06:56.108128 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:06:56.108128 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:06:56.108128 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:06:56.108128 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:06:56.108128 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:06:56.108128 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:06:56.108128 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:06:56.108128 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:06:56.108128 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:06:56.108128 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 36/70] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:06:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:06:56.108128 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a4e3a-2f07-44b2-b467-cf8bf8bee1c5 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:56.143379 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee8c9f-7dfc-4d63-a9db-4d2cd6bec4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:56.144343 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee8c9f-7dfc-4d63-a9db-4d2cd6bec4e0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:56.144633 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee8c9f-7dfc-4d63-a9db-4d2cd6bec4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:56.145691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee8c9f-7dfc-4d63-a9db-4d2cd6bec4e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:56.161235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee8c9f-7dfc-4d63-a9db-4d2cd6bec4e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:06:56.169659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-87ee8c9f-7dfc-4d63-a9db-4d2cd6bec4e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:06:56.169659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:06:56.169659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:06:56.169659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:06:56.169659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:06:56.169659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:06:56.169659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:06:56.169659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:06:56.169659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:06:56.169659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:06:56.169659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:06:56.169659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:06:56.169659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:06:56.169659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:06:56.169659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:06:56.169659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:06:56.170592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:06:56.170592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:06:56.170592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:06:56.170592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:06:56.170592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:06:56.170592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:06:56.170592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:06:56.170592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:06:56.170592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:06:56.170592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:06:56.170592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:06:56.170592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:06:56.170592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:06:56.170592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:06:56.170592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:06:56.170592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:06:56.171396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:06:56.171396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:06:56.171396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:06:56.171396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:06:56.171396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:06:56.171396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:06:56.171396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:06:56.171396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:06:56.171396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:06:56.171396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:06:56.171396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:06:56.171396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:06:56.171396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 35/71] 10.222.0.81 () {56 vars in 1114 bytes} [Tue Jun 7 14:06:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:06:56.172241 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a8387-fbe5-4779-a006-fa9f4a7b6c65 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:56.177739 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a8387-fbe5-4779-a006-fa9f4a7b6c65 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:56.177739 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a8387-fbe5-4779-a006-fa9f4a7b6c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:56.198064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-619a8387-fbe5-4779-a006-fa9f4a7b6c65 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:06:56.199104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-619a8387-fbe5-4779-a006-fa9f4a7b6c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:06:56.222321 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a4e3a-2f07-44b2-b467-cf8bf8bee1c5 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:56.256696 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d99460-972b-4205-8932-d442e76e1725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:56.257390 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d99460-972b-4205-8932-d442e76e1725 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:56.257674 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d99460-972b-4205-8932-d442e76e1725 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:56.257949 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d99460-972b-4205-8932-d442e76e1725 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:56.258692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d99460-972b-4205-8932-d442e76e1725 None admin] RBAC: Adding query filter params (name=member) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:06:56.258997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d99460-972b-4205-8932-d442e76e1725 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:56.261275 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d99460-972b-4205-8932-d442e76e1725 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:56.266915 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 36/72] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:06:56.267809 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a8387-fbe5-4779-a006-fa9f4a7b6c65 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:56.277949 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 37/73] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:06:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:06:56.278987 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d99460-972b-4205-8932-d442e76e1725 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:56.336038 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:56.351464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:56.367036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a8387-fbe5-4779-a006-fa9f4a7b6c65 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:56.380154 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d99460-972b-4205-8932-d442e76e1725 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:56.401660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52059b5-1d1a-43b3-a59a-1127a1704deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:56.402336 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52059b5-1d1a-43b3-a59a-1127a1704deb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:56.402658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52059b5-1d1a-43b3-a59a-1127a1704deb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:56.402934 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52059b5-1d1a-43b3-a59a-1127a1704deb None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:56.403723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52059b5-1d1a-43b3-a59a-1127a1704deb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:06:56.404029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52059b5-1d1a-43b3-a59a-1127a1704deb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:56.405424 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52059b5-1d1a-43b3-a59a-1127a1704deb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:56.413481 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 37/74] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:06:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:06:56.413797 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6705271-f78a-4fbd-b234-e268b0177712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:56.414473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6705271-f78a-4fbd-b234-e268b0177712 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:56.414630 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1d47f-8785-457b-aadf-96e007128deb None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:56.414791 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6705271-f78a-4fbd-b234-e268b0177712 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:56.414948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1d47f-8785-457b-aadf-96e007128deb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:56.415042 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6705271-f78a-4fbd-b234-e268b0177712 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:56.415182 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1d47f-8785-457b-aadf-96e007128deb None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:56.416344 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6705271-f78a-4fbd-b234-e268b0177712 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:06:56.416659 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6705271-f78a-4fbd-b234-e268b0177712 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:56.417394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 38/75] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:06:56.417911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1d47f-8785-457b-aadf-96e007128deb None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:56.422023 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6705271-f78a-4fbd-b234-e268b0177712 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:56.434195 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 38/76] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:06:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:06:56.434195 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6705271-f78a-4fbd-b234-e268b0177712 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:56.524707 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1d47f-8785-457b-aadf-96e007128deb None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:56.535473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6705271-f78a-4fbd-b234-e268b0177712 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:56.551993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:56.558719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b97d2c-8251-49e8-afd7-d2ed581d1a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:56.559673 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b97d2c-8251-49e8-afd7-d2ed581d1a7a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:56.559962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b97d2c-8251-49e8-afd7-d2ed581d1a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:56.560240 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b97d2c-8251-49e8-afd7-d2ed581d1a7a None admin] PATH_INFO: `/v3/users/d91da95dd8674152b0b2f5f61a71da6b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:56.569405 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44051e94-2c9b-4c1e-a221-c416a732502b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:56.570195 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44051e94-2c9b-4c1e-a221-c416a732502b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:56.570479 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44051e94-2c9b-4c1e-a221-c416a732502b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:56.570756 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44051e94-2c9b-4c1e-a221-c416a732502b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:56.572463 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44051e94-2c9b-4c1e-a221-c416a732502b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:56.574068 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44051e94-2c9b-4c1e-a221-c416a732502b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:56.574661 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b97d2c-8251-49e8-afd7-d2ed581d1a7a None admin] RBAC: Authorizing `identity:get_user(user_id=d91da95dd8674152b0b2f5f61a71da6b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:56.575722 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 39/77] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:06:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:06:56.576167 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b97d2c-8251-49e8-afd7-d2ed581d1a7a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:56.576487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44051e94-2c9b-4c1e-a221-c416a732502b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:56.577858 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 39/78] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:06:56 2022] GET /identity/v3/users/d91da95dd8674152b0b2f5f61a71da6b => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:06:56.579035 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9421b2-a043-4a49-8493-208ba395bf3d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:56.579354 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9421b2-a043-4a49-8493-208ba395bf3d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:56.579637 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9421b2-a043-4a49-8493-208ba395bf3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:56.585063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:56.596877 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-fc9421b2-a043-4a49-8493-208ba395bf3d None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:06:56.598086 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fc9421b2-a043-4a49-8493-208ba395bf3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:06:56.664684 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 40/79] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:06:56.664684 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9421b2-a043-4a49-8493-208ba395bf3d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:56.682220 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44051e94-2c9b-4c1e-a221-c416a732502b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:56.717926 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668a95e-309c-4b80-96a7-002a5b4a7431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:56.718805 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668a95e-309c-4b80-96a7-002a5b4a7431 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:56.719088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668a95e-309c-4b80-96a7-002a5b4a7431 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:56.719481 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668a95e-309c-4b80-96a7-002a5b4a7431 None admin] PATH_INFO: `/v3/users/2604a49f226a4bdf958c4047caf193e8` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:56.720691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5668a95e-309c-4b80-96a7-002a5b4a7431 None admin] RBAC: Authorizing `identity:get_user(user_id=2604a49f226a4bdf958c4047caf193e8)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:56.722150 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5668a95e-309c-4b80-96a7-002a5b4a7431 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:56.723809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 40/80] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:06:56 2022] GET /identity/v3/users/2604a49f226a4bdf958c4047caf193e8 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:06:56.724554 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668a95e-309c-4b80-96a7-002a5b4a7431 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:56.765138 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9421b2-a043-4a49-8493-208ba395bf3d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:56.780027 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:56.797895 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dfaebb-b067-44ed-a082-2b1b2184430b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:56.798779 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfaebb-b067-44ed-a082-2b1b2184430b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:56.799068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfaebb-b067-44ed-a082-2b1b2184430b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:56.799392 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfaebb-b067-44ed-a082-2b1b2184430b None admin] PATH_INFO: `/v3/users/d91da95dd8674152b0b2f5f61a71da6b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:56.800584 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dfaebb-b067-44ed-a082-2b1b2184430b None admin] RBAC: Authorizing `identity:get_user(user_id=d91da95dd8674152b0b2f5f61a71da6b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:56.802032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dfaebb-b067-44ed-a082-2b1b2184430b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:56.803907 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 41/81] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:06:56 2022] GET /identity/v3/users/d91da95dd8674152b0b2f5f61a71da6b => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:06:56.804506 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dfaebb-b067-44ed-a082-2b1b2184430b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:56.807574 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:56.829692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668a95e-309c-4b80-96a7-002a5b4a7431 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:56.862349 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eefdb5-dbe8-459f-955d-49c49c763858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:56.863118 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eefdb5-dbe8-459f-955d-49c49c763858 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:56.863502 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eefdb5-dbe8-459f-955d-49c49c763858 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:56.863793 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eefdb5-dbe8-459f-955d-49c49c763858 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:56.864509 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eefdb5-dbe8-459f-955d-49c49c763858 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:56.865950 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eefdb5-dbe8-459f-955d-49c49c763858 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:56.893015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 41/82] 10.222.0.81 () {60 vars in 1132 bytes} [Tue Jun 7 14:06:56 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:06:56.893875 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eefdb5-dbe8-459f-955d-49c49c763858 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:56.908573 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dfaebb-b067-44ed-a082-2b1b2184430b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:56.954117 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef800641-af90-4dfa-a2ed-514f86df8237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:56.955159 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef800641-af90-4dfa-a2ed-514f86df8237 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:56.955626 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef800641-af90-4dfa-a2ed-514f86df8237 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:56.956048 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef800641-af90-4dfa-a2ed-514f86df8237 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:57.010502 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eefdb5-dbe8-459f-955d-49c49c763858 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:57.022640 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:57.049555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef800641-af90-4dfa-a2ed-514f86df8237 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:57.051048 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef800641-af90-4dfa-a2ed-514f86df8237 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:57.051425 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf244f-fd12-43dd-afe6-68931d745165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:57.052242 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf244f-fd12-43dd-afe6-68931d745165 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:57.052848 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf244f-fd12-43dd-afe6-68931d745165 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:57.053202 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf244f-fd12-43dd-afe6-68931d745165 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:57.054743 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:57.066748 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bf244f-fd12-43dd-afe6-68931d745165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:06:57.075791 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-34bf244f-fd12-43dd-afe6-68931d745165 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:06:57.075791 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:06:57.075791 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:06:57.075791 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:06:57.075791 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:06:57.075791 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:06:57.075791 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:06:57.075791 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:06:57.075791 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:06:57.075791 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:06:57.075791 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:06:57.075791 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:06:57.075791 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:06:57.075791 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:06:57.075791 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:06:57.075791 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:06:57.076763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:06:57.076763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:06:57.076763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:06:57.076763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:06:57.076763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:06:57.076763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:06:57.076763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:06:57.076763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:06:57.076763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:06:57.076763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:06:57.076763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:06:57.076763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:06:57.076763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:06:57.076763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:06:57.076763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:06:57.076763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:06:57.083300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:06:57.083300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:06:57.083300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:06:57.083300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:06:57.083300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:06:57.083300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:06:57.083300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:06:57.083300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:06:57.083300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:06:57.083300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:06:57.083300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:06:57.083300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:06:57.083300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 42/83] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:06:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:06:57.083300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf244f-fd12-43dd-afe6-68931d745165 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:57.182506 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 42/84] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:06:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:06:57.183061 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef800641-af90-4dfa-a2ed-514f86df8237 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:57.212040 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf244f-fd12-43dd-afe6-68931d745165 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:57.251671 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5903d92-734e-404b-b2e7-2ff9304b5142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:57.252560 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5903d92-734e-404b-b2e7-2ff9304b5142 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:57.252986 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5903d92-734e-404b-b2e7-2ff9304b5142 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:57.253425 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5903d92-734e-404b-b2e7-2ff9304b5142 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:57.300755 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef800641-af90-4dfa-a2ed-514f86df8237 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:57.302711 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:57.306803 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:57.340239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d9df8-b9c2-4093-84c5-4417383ebb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:57.341693 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d9df8-b9c2-4093-84c5-4417383ebb27 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:57.342151 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d9df8-b9c2-4093-84c5-4417383ebb27 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:57.342564 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d9df8-b9c2-4093-84c5-4417383ebb27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:57.348737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5903d92-734e-404b-b2e7-2ff9304b5142 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:57.350243 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5903d92-734e-404b-b2e7-2ff9304b5142 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:57.423858 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68d9df8-b9c2-4093-84c5-4417383ebb27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:57.425597 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68d9df8-b9c2-4093-84c5-4417383ebb27 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:57.494486 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 43/85] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:06:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:06:57.495420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5903d92-734e-404b-b2e7-2ff9304b5142 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:57.538360 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 43/86] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:06:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:06:57.543264 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d9df8-b9c2-4093-84c5-4417383ebb27 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:57.543264 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:57.598955 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:57.632483 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5903d92-734e-404b-b2e7-2ff9304b5142 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:57.661352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d9df8-b9c2-4093-84c5-4417383ebb27 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:57.678005 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b9b29-9ef3-4292-ae88-8e22126cffa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:57.678685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b9b29-9ef3-4292-ae88-8e22126cffa5 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:57.678970 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b9b29-9ef3-4292-ae88-8e22126cffa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:57.679271 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b9b29-9ef3-4292-ae88-8e22126cffa5 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:57.680285 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b9b29-9ef3-4292-ae88-8e22126cffa5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:06:57.680593 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b9b29-9ef3-4292-ae88-8e22126cffa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:57.683034 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b9b29-9ef3-4292-ae88-8e22126cffa5 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:57.699465 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 44/87] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:06:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:06:57.699465 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b9b29-9ef3-4292-ae88-8e22126cffa5 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:57.705657 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4af0a-5091-4144-9e4a-c8dd42a6e931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:57.707118 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4af0a-5091-4144-9e4a-c8dd42a6e931 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:57.707635 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4af0a-5091-4144-9e4a-c8dd42a6e931 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:57.708129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4af0a-5091-4144-9e4a-c8dd42a6e931 None admin] PATH_INFO: `/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:57.709534 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4af0a-5091-4144-9e4a-c8dd42a6e931 None admin] RBAC: Authorizing `identity:get_project(project_id=a85ae30482264d7ba1c84ab0ca46d4e0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:57.711180 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4af0a-5091-4144-9e4a-c8dd42a6e931 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:57.713305 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 44/88] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:06:57 2022] GET /identity/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:06:57.714334 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4af0a-5091-4144-9e4a-c8dd42a6e931 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:57.787598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:57.804984 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b9b29-9ef3-4292-ae88-8e22126cffa5 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:57.822208 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4af0a-5091-4144-9e4a-c8dd42a6e931 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:57.835689 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:57.842912 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea22d7e-b801-43b1-83ef-48db8e75ab14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:57.843722 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea22d7e-b801-43b1-83ef-48db8e75ab14 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:57.844009 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea22d7e-b801-43b1-83ef-48db8e75ab14 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:57.844284 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea22d7e-b801-43b1-83ef-48db8e75ab14 None admin] PATH_INFO: `/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:57.845504 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea22d7e-b801-43b1-83ef-48db8e75ab14 None admin] RBAC: Authorizing `identity:get_project(project_id=a85ae30482264d7ba1c84ab0ca46d4e0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:57.847261 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea22d7e-b801-43b1-83ef-48db8e75ab14 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:57.848737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 45/89] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:06:57 2022] GET /identity/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:06:57.849448 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea22d7e-b801-43b1-83ef-48db8e75ab14 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:57.862988 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77408553-8bb1-42b2-8a4f-eb1503034563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:57.864215 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77408553-8bb1-42b2-8a4f-eb1503034563 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:57.864723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77408553-8bb1-42b2-8a4f-eb1503034563 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:57.865194 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77408553-8bb1-42b2-8a4f-eb1503034563 None admin] PATH_INFO: `/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:57.866630 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77408553-8bb1-42b2-8a4f-eb1503034563 None admin] RBAC: Authorizing `identity:get_project(project_id=a85ae30482264d7ba1c84ab0ca46d4e0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:57.868374 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77408553-8bb1-42b2-8a4f-eb1503034563 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:57.871250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 45/90] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:06:57 2022] GET /identity/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:06:57.872563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77408553-8bb1-42b2-8a4f-eb1503034563 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:57.960723 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea22d7e-b801-43b1-83ef-48db8e75ab14 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:57.983419 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77408553-8bb1-42b2-8a4f-eb1503034563 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:57.995579 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237ed6f-c7b4-475b-9387-6a7a2f173a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:57.996455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237ed6f-c7b4-475b-9387-6a7a2f173a24 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:57.996738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237ed6f-c7b4-475b-9387-6a7a2f173a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:57.997021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237ed6f-c7b4-475b-9387-6a7a2f173a24 None admin] PATH_INFO: `/v3/users/d91da95dd8674152b0b2f5f61a71da6b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:57.998569 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8237ed6f-c7b4-475b-9387-6a7a2f173a24 None admin] RBAC: Authorizing `identity:get_user(user_id=d91da95dd8674152b0b2f5f61a71da6b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:58.000026 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8237ed6f-c7b4-475b-9387-6a7a2f173a24 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:58.001764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 46/91] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:06:57 2022] GET /identity/v3/users/d91da95dd8674152b0b2f5f61a71da6b => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:06:58.002517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237ed6f-c7b4-475b-9387-6a7a2f173a24 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:58.023837 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26585d-110c-4dba-ac8f-2835133f9663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:58.024894 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26585d-110c-4dba-ac8f-2835133f9663 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:58.026056 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26585d-110c-4dba-ac8f-2835133f9663 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:58.026528 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26585d-110c-4dba-ac8f-2835133f9663 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:58.027613 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e26585d-110c-4dba-ac8f-2835133f9663 None admin] RBAC: Adding query filter params (role.id=ca91f64e7a46431c8aa4285c4620f216, scope.project.id=a85ae30482264d7ba1c84ab0ca46d4e0, user.id=d91da95dd8674152b0b2f5f61a71da6b) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:06:58.028207 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e26585d-110c-4dba-ac8f-2835133f9663 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:58.030012 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e26585d-110c-4dba-ac8f-2835133f9663 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:58.036745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 46/92] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:06:57 2022] GET /identity/v3/role_assignments?user.id=d91da95dd8674152b0b2f5f61a71da6b&scope.project.id=a85ae30482264d7ba1c84ab0ca46d4e0&role.id=ca91f64e7a46431c8aa4285c4620f216 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:06:58.038013 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26585d-110c-4dba-ac8f-2835133f9663 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:58.041015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:58.074228 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:58.111761 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237ed6f-c7b4-475b-9387-6a7a2f173a24 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:58.148014 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb1e44-c870-44e1-8496-5c5f861f303f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:58.148700 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb1e44-c870-44e1-8496-5c5f861f303f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:58.148988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb1e44-c870-44e1-8496-5c5f861f303f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:58.149641 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb1e44-c870-44e1-8496-5c5f861f303f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:58.150333 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb1e44-c870-44e1-8496-5c5f861f303f None admin] RBAC: Adding query filter params (role.id=82bbe3651b9b4f378fd150d93357b2b1, scope.project.id=a85ae30482264d7ba1c84ab0ca46d4e0, user.id=2604a49f226a4bdf958c4047caf193e8) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:06:58.150711 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb1e44-c870-44e1-8496-5c5f861f303f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:58.153935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb1e44-c870-44e1-8496-5c5f861f303f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:58.164344 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 47/93] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:06:58 2022] GET /identity/v3/role_assignments?user.id=2604a49f226a4bdf958c4047caf193e8&scope.project.id=a85ae30482264d7ba1c84ab0ca46d4e0&role.id=82bbe3651b9b4f378fd150d93357b2b1 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:06:58.164344 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb1e44-c870-44e1-8496-5c5f861f303f None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:58.169779 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26585d-110c-4dba-ac8f-2835133f9663 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:58.211863 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29dc893-a66c-4de3-b2c9-cb38e569ba96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:58.213553 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29dc893-a66c-4de3-b2c9-cb38e569ba96 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:58.214038 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29dc893-a66c-4de3-b2c9-cb38e569ba96 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:58.214576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29dc893-a66c-4de3-b2c9-cb38e569ba96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:58.215643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29dc893-a66c-4de3-b2c9-cb38e569ba96 None admin] RBAC: Adding query filter params (role.id=6d02db1fe03b447c9e519f4f2aa16574, scope.project.id=a85ae30482264d7ba1c84ab0ca46d4e0, user.id=d91da95dd8674152b0b2f5f61a71da6b) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:06:58.216226 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29dc893-a66c-4de3-b2c9-cb38e569ba96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:58.217872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29dc893-a66c-4de3-b2c9-cb38e569ba96 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:58.224976 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 47/94] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:06:58 2022] GET /identity/v3/role_assignments?user.id=d91da95dd8674152b0b2f5f61a71da6b&scope.project.id=a85ae30482264d7ba1c84ab0ca46d4e0&role.id=6d02db1fe03b447c9e519f4f2aa16574 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:06:58.304237 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb1e44-c870-44e1-8496-5c5f861f303f None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:58.356410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:58.359530 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb91965-01ea-4945-a9b5-299b2f8edd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:58.360224 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb91965-01ea-4945-a9b5-299b2f8edd29 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:58.360515 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb91965-01ea-4945-a9b5-299b2f8edd29 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:58.360792 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb91965-01ea-4945-a9b5-299b2f8edd29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:58.453365 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb91965-01ea-4945-a9b5-299b2f8edd29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:58.454903 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb91965-01ea-4945-a9b5-299b2f8edd29 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:58.584365 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 48/95] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:06:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:06:58.590243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29dc893-a66c-4de3-b2c9-cb38e569ba96 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:58.690534 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:58.736350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29dc893-a66c-4de3-b2c9-cb38e569ba96 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:58.786043 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a167740-ef2d-47d7-9125-934a413dd8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:58.786915 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a167740-ef2d-47d7-9125-934a413dd8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:58.787197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a167740-ef2d-47d7-9125-934a413dd8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:58.787500 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a167740-ef2d-47d7-9125-934a413dd8d5 None admin] PATH_INFO: `/v3/projects/48926fa411ae43869359fdc0695cd3a8` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:58.792190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a167740-ef2d-47d7-9125-934a413dd8d5 None admin] RBAC: Authorizing `identity:get_project(project_id=48926fa411ae43869359fdc0695cd3a8)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:58.793669 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a167740-ef2d-47d7-9125-934a413dd8d5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:58.795427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 48/96] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:06:58 2022] GET /identity/v3/projects/48926fa411ae43869359fdc0695cd3a8 => generated 312 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:06:58.801330 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb91965-01ea-4945-a9b5-299b2f8edd29 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:58.835385 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:58.919881 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb91965-01ea-4945-a9b5-299b2f8edd29 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:58.967957 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071adef-2549-4349-9f34-f57429aba48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:58.968805 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071adef-2549-4349-9f34-f57429aba48a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:58.969091 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071adef-2549-4349-9f34-f57429aba48a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:58.969366 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071adef-2549-4349-9f34-f57429aba48a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:58.970199 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071adef-2549-4349-9f34-f57429aba48a None admin] RBAC: Adding query filter params (role.id=ca91f64e7a46431c8aa4285c4620f216, scope.project.id=48926fa411ae43869359fdc0695cd3a8, user.id=d91da95dd8674152b0b2f5f61a71da6b) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:06:58.970509 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071adef-2549-4349-9f34-f57429aba48a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:06:58.971987 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071adef-2549-4349-9f34-f57429aba48a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:06:58.983620 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 49/97] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:06:58 2022] GET /identity/v3/role_assignments?user.id=d91da95dd8674152b0b2f5f61a71da6b&scope.project.id=48926fa411ae43869359fdc0695cd3a8&role.id=ca91f64e7a46431c8aa4285c4620f216 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:06:59.584945 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e89f37-16c8-42d1-8aaa-23e8068983a5 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:59.585341 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e89f37-16c8-42d1-8aaa-23e8068983a5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:59.585523 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e89f37-16c8-42d1-8aaa-23e8068983a5 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:59.586886 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 49/98] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:06:59.592698 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9e098-a525-46ea-884b-028d609f88a3 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:59.592995 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9e098-a525-46ea-884b-028d609f88a3 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:59.593270 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9e098-a525-46ea-884b-028d609f88a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:59.620496 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-8df9e098-a525-46ea-884b-028d609f88a3 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:06:59.621712 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8df9e098-a525-46ea-884b-028d609f88a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:06:59.714232 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 50/99] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:06:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:06:59.745891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e89f37-16c8-42d1-8aaa-23e8068983a5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:59.880588 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e89f37-16c8-42d1-8aaa-23e8068983a5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:06:59.921582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1c6ea-2c5f-4b47-974a-9221281c5890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:06:59.922384 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1c6ea-2c5f-4b47-974a-9221281c5890 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:06:59.922668 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1c6ea-2c5f-4b47-974a-9221281c5890 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:06:59.922940 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1c6ea-2c5f-4b47-974a-9221281c5890 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:06:59.925124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:06:59.936365 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1c6ea-2c5f-4b47-974a-9221281c5890 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:06:59.944308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-26c1c6ea-2c5f-4b47-974a-9221281c5890 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:06:59.944308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:06:59.944308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:06:59.944308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:06:59.944308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:06:59.944308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:06:59.944308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:06:59.944308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:06:59.944308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:06:59.944308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:06:59.944308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:06:59.944308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:06:59.944308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:06:59.944308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:06:59.944308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:06:59.944308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:06:59.945243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:06:59.945243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:06:59.945243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:06:59.945243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:06:59.945243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:06:59.945243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:06:59.945243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:06:59.945243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:06:59.945243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:06:59.945243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:06:59.945243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:06:59.945243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:06:59.945243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:06:59.945243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:06:59.945243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:06:59.945243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:06:59.946014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:06:59.946014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:06:59.946014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:06:59.946014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:06:59.946014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:06:59.946014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:06:59.946014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:06:59.946014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:06:59.946014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:06:59.946014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:06:59.946014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:06:59.946014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:06:59.946014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 50/100] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:06:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:06:59.952926 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9e098-a525-46ea-884b-028d609f88a3 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:06:59.997129 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:00.091859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9e098-a525-46ea-884b-028d609f88a3 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:00.144904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b32970-1d41-4dfd-bc62-03ce31c0f9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:00.145516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b32970-1d41-4dfd-bc62-03ce31c0f9c8 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:00.145807 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b32970-1d41-4dfd-bc62-03ce31c0f9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:00.146087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b32970-1d41-4dfd-bc62-03ce31c0f9c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:00.146855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b32970-1d41-4dfd-bc62-03ce31c0f9c8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:00.147165 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b32970-1d41-4dfd-bc62-03ce31c0f9c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:00.148620 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b32970-1d41-4dfd-bc62-03ce31c0f9c8 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:00.173613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 51/101] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:06:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:00.179619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1c6ea-2c5f-4b47-974a-9221281c5890 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:00.195283 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d0306-aebe-43d5-b1d9-deece3557cfc None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:00.195283 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d0306-aebe-43d5-b1d9-deece3557cfc None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:00.195283 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d0306-aebe-43d5-b1d9-deece3557cfc None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:00.195283 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 52/102] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:07:00.197346 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec46d6-2bac-42ab-9fbf-d206bdfc4a67 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:00.197668 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec46d6-2bac-42ab-9fbf-d206bdfc4a67 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:00.198151 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec46d6-2bac-42ab-9fbf-d206bdfc4a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:00.219243 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-49ec46d6-2bac-42ab-9fbf-d206bdfc4a67 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:00.219243 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-49ec46d6-2bac-42ab-9fbf-d206bdfc4a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:00.301145 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 53/103] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:07:00.340490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec46d6-2bac-42ab-9fbf-d206bdfc4a67 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:00.366564 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1c6ea-2c5f-4b47-974a-9221281c5890 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:00.375350 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:00.425400 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f510073f-41c1-4f4a-92f6-80f0c713530e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:00.426205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510073f-41c1-4f4a-92f6-80f0c713530e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:00.426497 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510073f-41c1-4f4a-92f6-80f0c713530e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:00.426780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510073f-41c1-4f4a-92f6-80f0c713530e None admin] PATH_INFO: `/v3/users/b628faa9048645e18a23946e0e07b058` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:00.456040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f510073f-41c1-4f4a-92f6-80f0c713530e None admin] RBAC: Authorizing `identity:get_user(user_id=b628faa9048645e18a23946e0e07b058)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:00.457535 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f510073f-41c1-4f4a-92f6-80f0c713530e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:00.459615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 51/104] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:00 2022] GET /identity/v3/users/b628faa9048645e18a23946e0e07b058 => generated 289 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:00.468556 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f510073f-41c1-4f4a-92f6-80f0c713530e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:00.526295 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec46d6-2bac-42ab-9fbf-d206bdfc4a67 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:00.577324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:00.586644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc91ab-d4fc-4907-ab0d-39db37a610bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:00.587664 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc91ab-d4fc-4907-ab0d-39db37a610bc None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:00.588171 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc91ab-d4fc-4907-ab0d-39db37a610bc None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:00.588466 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc91ab-d4fc-4907-ab0d-39db37a610bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:00.592052 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbc91ab-d4fc-4907-ab0d-39db37a610bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:00.593779 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbc91ab-d4fc-4907-ab0d-39db37a610bc None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:00.595536 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 54/105] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:07:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:00.604617 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc91ab-d4fc-4907-ab0d-39db37a610bc None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:00.640777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f510073f-41c1-4f4a-92f6-80f0c713530e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:00.731930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050212b5-cad0-4218-be50-6fef4897aff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:00.732605 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050212b5-cad0-4218-be50-6fef4897aff2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:00.732897 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050212b5-cad0-4218-be50-6fef4897aff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:00.733175 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050212b5-cad0-4218-be50-6fef4897aff2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:00.786403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:00.823193 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc91ab-d4fc-4907-ab0d-39db37a610bc None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:00.865950 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050212b5-cad0-4218-be50-6fef4897aff2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:00.876911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050212b5-cad0-4218-be50-6fef4897aff2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:00.912512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12bb97d-e8ad-4944-915d-1c4e15ef7bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:00.913573 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12bb97d-e8ad-4944-915d-1c4e15ef7bc2 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:00.914059 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12bb97d-e8ad-4944-915d-1c4e15ef7bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:00.914531 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12bb97d-e8ad-4944-915d-1c4e15ef7bc2 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:00.915991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12bb97d-e8ad-4944-915d-1c4e15ef7bc2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:00.917811 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12bb97d-e8ad-4944-915d-1c4e15ef7bc2 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:00.970198 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:00.977809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 55/106] 10.222.0.81 () {60 vars in 1141 bytes} [Tue Jun 7 14:07:00 2022] POST /identity/v3/projects => generated 302 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:07:01.060019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 52/107] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 14:07:01.066338 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12bb97d-e8ad-4944-915d-1c4e15ef7bc2 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:01.230411 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12bb97d-e8ad-4944-915d-1c4e15ef7bc2 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:01.272308 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1457f4-e177-46e9-8bf6-e6bc94ae27e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:01.273216 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1457f4-e177-46e9-8bf6-e6bc94ae27e4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:01.273652 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1457f4-e177-46e9-8bf6-e6bc94ae27e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:01.273945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1457f4-e177-46e9-8bf6-e6bc94ae27e4 None admin] PATH_INFO: `/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:01.275828 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1457f4-e177-46e9-8bf6-e6bc94ae27e4 None admin] RBAC: Authorizing `identity:get_project(project_id=a85ae30482264d7ba1c84ab0ca46d4e0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:01.277313 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1457f4-e177-46e9-8bf6-e6bc94ae27e4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:01.283356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 56/108] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:01 2022] GET /identity/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:01.288358 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050212b5-cad0-4218-be50-6fef4897aff2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:01.442777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050212b5-cad0-4218-be50-6fef4897aff2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:01.477774 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc997f-cbbb-45fd-8e73-8d0d4939a303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:01.479200 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc997f-cbbb-45fd-8e73-8d0d4939a303 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:01.479743 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc997f-cbbb-45fd-8e73-8d0d4939a303 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:01.480223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc997f-cbbb-45fd-8e73-8d0d4939a303 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:01.481285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc997f-cbbb-45fd-8e73-8d0d4939a303 None admin] RBAC: Adding query filter params (role.id=a1d21c4d26ef419298e001001f4655b7, scope.project.id=a85ae30482264d7ba1c84ab0ca46d4e0, user.id=b628faa9048645e18a23946e0e07b058) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:01.481936 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc997f-cbbb-45fd-8e73-8d0d4939a303 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:01.483619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc997f-cbbb-45fd-8e73-8d0d4939a303 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:01.489817 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 53/109] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:07:01 2022] GET /identity/v3/role_assignments?user.id=b628faa9048645e18a23946e0e07b058&scope.project.id=a85ae30482264d7ba1c84ab0ca46d4e0&role.id=a1d21c4d26ef419298e001001f4655b7 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:01.682036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd21ed-6d0e-4535-b44c-955297e07816 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:01.682431 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd21ed-6d0e-4535-b44c-955297e07816 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:01.682617 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd21ed-6d0e-4535-b44c-955297e07816 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:01.684027 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 57/110] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:01.690152 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac023d3-3f7a-4367-851e-31e0f441949c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:01.690444 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac023d3-3f7a-4367-851e-31e0f441949c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:01.690724 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac023d3-3f7a-4367-851e-31e0f441949c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:01.716368 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3ac023d3-3f7a-4367-851e-31e0f441949c None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:01.717684 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3ac023d3-3f7a-4367-851e-31e0f441949c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:01.731087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:01.738708 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d3d15-ae53-410d-bb97-39358b6ede9d None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:01.739389 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d3d15-ae53-410d-bb97-39358b6ede9d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:01.739890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d3d15-ae53-410d-bb97-39358b6ede9d None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:01.741535 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 58/111] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:07:01.748363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625d308-c31c-497d-afab-def327c16148 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:01.748971 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625d308-c31c-497d-afab-def327c16148 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:01.749486 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625d308-c31c-497d-afab-def327c16148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:01.768457 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:01.784012 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-5625d308-c31c-497d-afab-def327c16148 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:01.785603 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5625d308-c31c-497d-afab-def327c16148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:01.830979 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 54/112] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:01.896656 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 59/113] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:07:01.911165 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac023d3-3f7a-4367-851e-31e0f441949c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:01.935510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e499f-dcc0-41e8-8635-64ebcc40ce64 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:01.935845 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e499f-dcc0-41e8-8635-64ebcc40ce64 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:01.936092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e499f-dcc0-41e8-8635-64ebcc40ce64 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:01.937489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 60/114] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:07:01.943457 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9199672-850b-4979-80b0-ab47ca459e92 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:01.943751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9199672-850b-4979-80b0-ab47ca459e92 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:01.944039 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9199672-850b-4979-80b0-ab47ca459e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:01.963386 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a9199672-850b-4979-80b0-ab47ca459e92 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:01.964909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a9199672-850b-4979-80b0-ab47ca459e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:02.043119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac023d3-3f7a-4367-851e-31e0f441949c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:02.094049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4d838-2b02-40d7-9123-83466dce21ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:02.094931 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4d838-2b02-40d7-9123-83466dce21ee None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:02.095361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4d838-2b02-40d7-9123-83466dce21ee None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:02.095702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4d838-2b02-40d7-9123-83466dce21ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:02.108920 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d4d838-2b02-40d7-9123-83466dce21ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:02.118426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-17d4d838-2b02-40d7-9123-83466dce21ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:07:02.118426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:02.118426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:02.118426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:02.118426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:02.118426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:02.118426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:02.118426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:02.118426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:02.118426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:02.118426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:02.118426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:02.118426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:02.118426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:02.118426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:02.118426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:02.119627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:02.119627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:02.119627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:02.119627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:02.119627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:02.119627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:02.119627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:02.119627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:02.119627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:02.119627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:02.119627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:02.119627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:02.119627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:02.119627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:02.119627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:02.119627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:02.120437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:02.120437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:02.120437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:02.120437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:02.120437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:02.120437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:02.120437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:02.120437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:02.120437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:02.120437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:02.120437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:07:02.120437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:02.120437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 55/115] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:07:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:02.121707 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4d838-2b02-40d7-9123-83466dce21ee None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:02.132173 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:02.133311 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 61/116] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:02.134086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9199672-850b-4979-80b0-ab47ca459e92 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:02.302238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:02.302536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4d838-2b02-40d7-9123-83466dce21ee None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:02.346781 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df2017-f6d8-4fca-8ec2-2ac6900a933a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:02.347610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df2017-f6d8-4fca-8ec2-2ac6900a933a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:02.347892 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df2017-f6d8-4fca-8ec2-2ac6900a933a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:02.348165 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df2017-f6d8-4fca-8ec2-2ac6900a933a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:02.364949 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df2017-f6d8-4fca-8ec2-2ac6900a933a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:02.373523 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-b4df2017-f6d8-4fca-8ec2-2ac6900a933a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:02.373523 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:02.373523 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:02.373523 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:02.373523 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:02.373523 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:02.373523 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:02.373523 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:02.373523 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:02.373523 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:02.373523 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:02.373523 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:02.373523 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:02.373523 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:02.373523 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:02.373523 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:02.374383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:02.374383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:02.374383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:02.374383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:02.374383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:02.374383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:02.374383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:02.374383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:02.374383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:02.374383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:02.374383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:02.374383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:02.374383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:02.374383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:02.374383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:02.374383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:02.375136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:02.375136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:02.375136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:02.375136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:02.375136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:02.375136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:02.375136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:02.375136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:02.375136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:02.375136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:02.375136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:02.375136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:02.375136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 56/117] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:02.376651 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df2017-f6d8-4fca-8ec2-2ac6900a933a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:02.389164 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9199672-850b-4979-80b0-ab47ca459e92 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:02.461952 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba50fb-a85c-41dc-b517-b679194441c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:02.463148 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba50fb-a85c-41dc-b517-b679194441c3 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:02.464064 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba50fb-a85c-41dc-b517-b679194441c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:02.465530 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba50fb-a85c-41dc-b517-b679194441c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:02.466664 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba50fb-a85c-41dc-b517-b679194441c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:02.471670 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba50fb-a85c-41dc-b517-b679194441c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:02.473112 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba50fb-a85c-41dc-b517-b679194441c3 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:02.480519 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:02.489635 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 62/118] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:02.490303 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba50fb-a85c-41dc-b517-b679194441c3 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:02.523956 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df2017-f6d8-4fca-8ec2-2ac6900a933a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:02.563423 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13977fda-2e7b-4d61-801f-f6710a1f43a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:02.564868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13977fda-2e7b-4d61-801f-f6710a1f43a5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:02.565547 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13977fda-2e7b-4d61-801f-f6710a1f43a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:02.566024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13977fda-2e7b-4d61-801f-f6710a1f43a5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:02.579568 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13977fda-2e7b-4d61-801f-f6710a1f43a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:02.588718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-13977fda-2e7b-4d61-801f-f6710a1f43a5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:07:02.588718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:02.588718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:02.588718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:02.588718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:02.588718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:02.588718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:02.588718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:02.588718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:02.588718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:02.588718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:02.588718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:02.588718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:02.588718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:02.588718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:02.588718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:02.589589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:02.589589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:02.589589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:02.589589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:02.589589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:02.589589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:02.589589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:02.589589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:02.589589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:02.589589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:02.589589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:02.589589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:02.589589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:02.589589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:02.589589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:02.589589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:02.590370 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:02.590370 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:02.590370 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:02.590370 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:02.590370 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:02.590370 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:02.590370 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:02.590370 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:02.590370 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:02.590370 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:02.590370 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:07:02.590370 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:02.593757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 57/119] 10.222.0.81 () {56 vars in 1114 bytes} [Tue Jun 7 14:07:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:02.594857 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13977fda-2e7b-4d61-801f-f6710a1f43a5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:02.689698 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba50fb-a85c-41dc-b517-b679194441c3 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:02.719543 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:02.729767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25982b4-6c1f-4d3d-aedb-dfc00dee6341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:02.730442 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25982b4-6c1f-4d3d-aedb-dfc00dee6341 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:02.730729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25982b4-6c1f-4d3d-aedb-dfc00dee6341 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:02.731007 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25982b4-6c1f-4d3d-aedb-dfc00dee6341 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:02.731787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25982b4-6c1f-4d3d-aedb-dfc00dee6341 None admin] RBAC: Adding query filter params (name=member) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:02.732094 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25982b4-6c1f-4d3d-aedb-dfc00dee6341 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:02.733694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25982b4-6c1f-4d3d-aedb-dfc00dee6341 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:02.742910 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 63/120] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:07:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:02.744649 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13977fda-2e7b-4d61-801f-f6710a1f43a5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:02.744773 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25982b4-6c1f-4d3d-aedb-dfc00dee6341 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:02.775831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:02.787572 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c78845-140e-49e0-b1c4-a671fc92b299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:02.788567 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c78845-140e-49e0-b1c4-a671fc92b299 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:02.788949 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c78845-140e-49e0-b1c4-a671fc92b299 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:02.789322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c78845-140e-49e0-b1c4-a671fc92b299 None admin] PATH_INFO: `/v3/users/2604a49f226a4bdf958c4047caf193e8` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:02.791184 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c78845-140e-49e0-b1c4-a671fc92b299 None admin] RBAC: Authorizing `identity:get_user(user_id=2604a49f226a4bdf958c4047caf193e8)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:02.793359 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c78845-140e-49e0-b1c4-a671fc92b299 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:02.796016 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 58/121] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:02 2022] GET /identity/v3/users/2604a49f226a4bdf958c4047caf193e8 => generated 247 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:02.797706 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dfd5d-0144-4e2e-9066-1522ce627470 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:02.798116 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dfd5d-0144-4e2e-9066-1522ce627470 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:02.798492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dfd5d-0144-4e2e-9066-1522ce627470 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:02.800704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 59/122] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:02.801842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131dfd5d-0144-4e2e-9066-1522ce627470 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:02.861455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25982b4-6c1f-4d3d-aedb-dfc00dee6341 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:02.899565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f79b40-39aa-45d5-ab2b-c3d1cb33042d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:02.901104 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f79b40-39aa-45d5-ab2b-c3d1cb33042d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:02.901590 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f79b40-39aa-45d5-ab2b-c3d1cb33042d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:02.902054 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f79b40-39aa-45d5-ab2b-c3d1cb33042d None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:02.902978 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f79b40-39aa-45d5-ab2b-c3d1cb33042d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:02.903571 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f79b40-39aa-45d5-ab2b-c3d1cb33042d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:02.905245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f79b40-39aa-45d5-ab2b-c3d1cb33042d None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:02.914967 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 64/123] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:07:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:02.916574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f79b40-39aa-45d5-ab2b-c3d1cb33042d None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:02.939338 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131dfd5d-0144-4e2e-9066-1522ce627470 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:02.982385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d809a771-a7c7-458d-9d57-3fd0e59365bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:02.983511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:02.986484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809a771-a7c7-458d-9d57-3fd0e59365bd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:02.987016 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809a771-a7c7-458d-9d57-3fd0e59365bd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:02.987620 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809a771-a7c7-458d-9d57-3fd0e59365bd None admin] PATH_INFO: `/v3/users/d91da95dd8674152b0b2f5f61a71da6b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:02.989368 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d809a771-a7c7-458d-9d57-3fd0e59365bd None admin] RBAC: Authorizing `identity:get_user(user_id=d91da95dd8674152b0b2f5f61a71da6b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:02.991421 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d809a771-a7c7-458d-9d57-3fd0e59365bd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:02.993243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 60/124] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:02 2022] GET /identity/v3/users/d91da95dd8674152b0b2f5f61a71da6b => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:03.005287 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce338bc-5373-4dcd-b8ce-9ae77c61fc87 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:03.005287 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce338bc-5373-4dcd-b8ce-9ae77c61fc87 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:03.005287 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce338bc-5373-4dcd-b8ce-9ae77c61fc87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:03.020266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1ce338bc-5373-4dcd-b8ce-9ae77c61fc87 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:03.021944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1ce338bc-5373-4dcd-b8ce-9ae77c61fc87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:03.043073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f79b40-39aa-45d5-ab2b-c3d1cb33042d None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:03.078356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320960c1-2786-4e75-9e7e-15cc3238f420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:03.079731 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320960c1-2786-4e75-9e7e-15cc3238f420 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:03.080093 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320960c1-2786-4e75-9e7e-15cc3238f420 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:03.080631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320960c1-2786-4e75-9e7e-15cc3238f420 None admin] PATH_INFO: `/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:03.081896 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320960c1-2786-4e75-9e7e-15cc3238f420 None admin] RBAC: Authorizing `identity:get_project(project_id=a85ae30482264d7ba1c84ab0ca46d4e0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:03.083841 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320960c1-2786-4e75-9e7e-15cc3238f420 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:03.085580 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 65/125] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:02 2022] GET /identity/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:03.086371 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320960c1-2786-4e75-9e7e-15cc3238f420 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:03.090788 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:03.104897 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 61/126] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:03.111589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce338bc-5373-4dcd-b8ce-9ae77c61fc87 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:03.208430 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320960c1-2786-4e75-9e7e-15cc3238f420 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:03.234234 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:03.239828 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce338bc-5373-4dcd-b8ce-9ae77c61fc87 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:03.251406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0afdb9c-ade1-4574-b1ef-17c3e6d7988e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:03.252615 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afdb9c-ade1-4574-b1ef-17c3e6d7988e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:03.252907 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afdb9c-ade1-4574-b1ef-17c3e6d7988e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:03.253202 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afdb9c-ade1-4574-b1ef-17c3e6d7988e None admin] PATH_INFO: `/v3/users/d91da95dd8674152b0b2f5f61a71da6b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:03.254688 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0afdb9c-ade1-4574-b1ef-17c3e6d7988e None admin] RBAC: Authorizing `identity:get_user(user_id=d91da95dd8674152b0b2f5f61a71da6b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:03.256340 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0afdb9c-ade1-4574-b1ef-17c3e6d7988e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:03.263285 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 66/127] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:03 2022] GET /identity/v3/users/d91da95dd8674152b0b2f5f61a71da6b => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:03.265225 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0afdb9c-ade1-4574-b1ef-17c3e6d7988e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:03.283524 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b8035-b48b-4d0f-819f-d2e3ad92012e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:03.284347 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b8035-b48b-4d0f-819f-d2e3ad92012e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:03.284637 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b8035-b48b-4d0f-819f-d2e3ad92012e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:03.284960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b8035-b48b-4d0f-819f-d2e3ad92012e None admin] PATH_INFO: `/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:03.286105 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702b8035-b48b-4d0f-819f-d2e3ad92012e None admin] RBAC: Authorizing `identity:get_project(project_id=a85ae30482264d7ba1c84ab0ca46d4e0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:03.288197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702b8035-b48b-4d0f-819f-d2e3ad92012e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:03.289718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 62/128] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:03 2022] GET /identity/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:03.291047 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b8035-b48b-4d0f-819f-d2e3ad92012e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:03.360181 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:03.405533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0afdb9c-ade1-4574-b1ef-17c3e6d7988e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:03.406677 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b8035-b48b-4d0f-819f-d2e3ad92012e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:03.442970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18820881-7292-4aba-94bd-80fd4d315479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:03.443758 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18820881-7292-4aba-94bd-80fd4d315479 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:03.444048 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18820881-7292-4aba-94bd-80fd4d315479 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:03.444330 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18820881-7292-4aba-94bd-80fd4d315479 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:03.453412 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497960e9-b2c7-4130-b510-86f125a13acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:03.454608 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497960e9-b2c7-4130-b510-86f125a13acc None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:03.455388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497960e9-b2c7-4130-b510-86f125a13acc None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:03.455678 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18820881-7292-4aba-94bd-80fd4d315479 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:03.456089 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497960e9-b2c7-4130-b510-86f125a13acc None admin] PATH_INFO: `/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0/users/2604a49f226a4bdf958c4047caf193e8/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:03.458464 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497960e9-b2c7-4130-b510-86f125a13acc None admin] RBAC: Authorizing `identity:create_grant(project_id=a85ae30482264d7ba1c84ab0ca46d4e0, user_id=2604a49f226a4bdf958c4047caf193e8, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:03.460176 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497960e9-b2c7-4130-b510-86f125a13acc None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:03.463705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-18820881-7292-4aba-94bd-80fd4d315479 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:03.463705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:03.463705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:03.463705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:03.463705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:03.463705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:03.463705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:03.463705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:03.463705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:03.463705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:03.463705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:03.463705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:03.463705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:03.463705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:03.463705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:03.463705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:03.464582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:03.464582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:03.464582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:03.464582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:03.464582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:03.464582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:03.464582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:03.464582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:03.464582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:03.464582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:03.464582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:03.464582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:03.464582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:03.464582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:03.464582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:03.464582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:03.465367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:03.465367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:03.465367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:03.465367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:03.465367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:03.465367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:03.465367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:03.465367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:03.465367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:03.465367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:03.465367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:03.465367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:03.465987 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 63/129] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:03.471778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18820881-7292-4aba-94bd-80fd4d315479 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:03.486020 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 67/130] 10.222.0.81 () {58 vars in 1439 bytes} [Tue Jun 7 14:07:03 2022] PUT /identity/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0/users/2604a49f226a4bdf958c4047caf193e8/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:07:03.487023 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497960e9-b2c7-4130-b510-86f125a13acc None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:03.551779 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:03.585896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18820881-7292-4aba-94bd-80fd4d315479 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:03.608773 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:03.621106 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497960e9-b2c7-4130-b510-86f125a13acc None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:03.626160 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68730543-99d2-4402-ae03-04ce5c77a163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:03.627263 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68730543-99d2-4402-ae03-04ce5c77a163 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:03.627645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68730543-99d2-4402-ae03-04ce5c77a163 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:03.628029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68730543-99d2-4402-ae03-04ce5c77a163 None admin] PATH_INFO: `/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:03.629524 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68730543-99d2-4402-ae03-04ce5c77a163 None admin] RBAC: Authorizing `identity:get_project(project_id=a85ae30482264d7ba1c84ab0ca46d4e0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:03.631129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68730543-99d2-4402-ae03-04ce5c77a163 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:03.633372 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 64/131] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:03 2022] GET /identity/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:03.634350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68730543-99d2-4402-ae03-04ce5c77a163 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:03.663038 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a6d3d-6c09-441f-bc90-4b4ce7b3cb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:03.664516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a6d3d-6c09-441f-bc90-4b4ce7b3cb33 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:03.664935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a6d3d-6c09-441f-bc90-4b4ce7b3cb33 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:03.665391 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a6d3d-6c09-441f-bc90-4b4ce7b3cb33 None admin] PATH_INFO: `/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0/users/d91da95dd8674152b0b2f5f61a71da6b/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:03.668302 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a6d3d-6c09-441f-bc90-4b4ce7b3cb33 None admin] RBAC: Authorizing `identity:create_grant(project_id=a85ae30482264d7ba1c84ab0ca46d4e0, user_id=d91da95dd8674152b0b2f5f61a71da6b, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:03.670022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a6d3d-6c09-441f-bc90-4b4ce7b3cb33 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:03.690875 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 68/132] 10.222.0.81 () {58 vars in 1439 bytes} [Tue Jun 7 14:07:03 2022] PUT /identity/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0/users/d91da95dd8674152b0b2f5f61a71da6b/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:07:03.692270 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a6d3d-6c09-441f-bc90-4b4ce7b3cb33 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:03.781557 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68730543-99d2-4402-ae03-04ce5c77a163 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:03.821981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a6d3d-6c09-441f-bc90-4b4ce7b3cb33 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:03.833021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:03.867189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb712a-8a45-44fc-8434-19e99e2d1bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:03.868511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb712a-8a45-44fc-8434-19e99e2d1bfa None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:03.869715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb712a-8a45-44fc-8434-19e99e2d1bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:03.870300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb712a-8a45-44fc-8434-19e99e2d1bfa None admin] PATH_INFO: `/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0/users/d91da95dd8674152b0b2f5f61a71da6b/roles/6d02db1fe03b447c9e519f4f2aa16574` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:03.873473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb712a-8a45-44fc-8434-19e99e2d1bfa None admin] RBAC: Authorizing `identity:create_grant(project_id=a85ae30482264d7ba1c84ab0ca46d4e0, user_id=d91da95dd8674152b0b2f5f61a71da6b, role_id=6d02db1fe03b447c9e519f4f2aa16574)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:03.875192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb712a-8a45-44fc-8434-19e99e2d1bfa None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:03.888592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6188e230-a511-4c90-9098-f81a987c9b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:03.889329 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188e230-a511-4c90-9098-f81a987c9b94 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:03.889679 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188e230-a511-4c90-9098-f81a987c9b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:03.890141 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188e230-a511-4c90-9098-f81a987c9b94 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:03.890793 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6188e230-a511-4c90-9098-f81a987c9b94 None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:03.891109 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6188e230-a511-4c90-9098-f81a987c9b94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:03.892619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6188e230-a511-4c90-9098-f81a987c9b94 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:03.895767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 69/133] 10.222.0.81 () {58 vars in 1439 bytes} [Tue Jun 7 14:07:03 2022] PUT /identity/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0/users/d91da95dd8674152b0b2f5f61a71da6b/roles/6d02db1fe03b447c9e519f4f2aa16574 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:07:03.912016 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 65/134] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:07:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:03.918905 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb712a-8a45-44fc-8434-19e99e2d1bfa None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:04.062981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb712a-8a45-44fc-8434-19e99e2d1bfa None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:04.103905 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58953290-db11-4e6b-9468-6e8d256099d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:04.105175 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58953290-db11-4e6b-9468-6e8d256099d2 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:04.105599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58953290-db11-4e6b-9468-6e8d256099d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:04.106012 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58953290-db11-4e6b-9468-6e8d256099d2 None admin] PATH_INFO: `/v3/users/d91da95dd8674152b0b2f5f61a71da6b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:04.107567 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58953290-db11-4e6b-9468-6e8d256099d2 None admin] RBAC: Authorizing `identity:get_user(user_id=d91da95dd8674152b0b2f5f61a71da6b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:04.109227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58953290-db11-4e6b-9468-6e8d256099d2 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:04.111064 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 70/135] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:03 2022] GET /identity/v3/users/d91da95dd8674152b0b2f5f61a71da6b => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:04.116873 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6188e230-a511-4c90-9098-f81a987c9b94 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:04.206824 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:04.267617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6188e230-a511-4c90-9098-f81a987c9b94 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:04.311118 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66a683-08d7-4cb3-b7b8-9d47a06e4c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:04.312023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66a683-08d7-4cb3-b7b8-9d47a06e4c7f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:04.312310 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66a683-08d7-4cb3-b7b8-9d47a06e4c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:04.312589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66a683-08d7-4cb3-b7b8-9d47a06e4c7f None admin] PATH_INFO: `/v3/projects/48926fa411ae43869359fdc0695cd3a8` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:04.317896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e66a683-08d7-4cb3-b7b8-9d47a06e4c7f None admin] RBAC: Authorizing `identity:get_project(project_id=48926fa411ae43869359fdc0695cd3a8)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:04.319414 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e66a683-08d7-4cb3-b7b8-9d47a06e4c7f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:04.321384 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 66/136] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:04 2022] GET /identity/v3/projects/48926fa411ae43869359fdc0695cd3a8 => generated 312 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:04.327331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58953290-db11-4e6b-9468-6e8d256099d2 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:04.354848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903dbf5f-ef1f-4f44-bb9d-6e793211239b None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:04.355166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903dbf5f-ef1f-4f44-bb9d-6e793211239b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:04.359449 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903dbf5f-ef1f-4f44-bb9d-6e793211239b None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:04.359449 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 67/137] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:04.362228 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5240ca-e440-40ca-aaf7-c0fda871202c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:04.362520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5240ca-e440-40ca-aaf7-c0fda871202c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:04.362800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5240ca-e440-40ca-aaf7-c0fda871202c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:04.386385 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:04.400418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7b5240ca-e440-40ca-aaf7-c0fda871202c None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:04.401761 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7b5240ca-e440-40ca-aaf7-c0fda871202c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:04.473055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 68/138] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:07:04.474870 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b569ab-0ce2-4364-9de4-ce7ae0ecb079 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:04.475168 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b569ab-0ce2-4364-9de4-ce7ae0ecb079 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:04.475493 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b569ab-0ce2-4364-9de4-ce7ae0ecb079 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:04.482056 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 69/139] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:04.484281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e7eed-e64d-4eda-9524-461b18ca1559 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:04.484690 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e7eed-e64d-4eda-9524-461b18ca1559 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:04.485119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e7eed-e64d-4eda-9524-461b18ca1559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:04.504761 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d79e7eed-e64d-4eda-9524-461b18ca1559 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:04.507090 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d79e7eed-e64d-4eda-9524-461b18ca1559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:04.542517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58953290-db11-4e6b-9468-6e8d256099d2 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:04.582441 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:04.585010 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ce3fc-cb6f-4882-8584-0e0a837fcdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:04.585978 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ce3fc-cb6f-4882-8584-0e0a837fcdc8 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:04.586264 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ce3fc-cb6f-4882-8584-0e0a837fcdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:04.586539 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ce3fc-cb6f-4882-8584-0e0a837fcdc8 None admin] PATH_INFO: `/v3/projects/48926fa411ae43869359fdc0695cd3a8/users/d91da95dd8674152b0b2f5f61a71da6b/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:04.589970 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ce3fc-cb6f-4882-8584-0e0a837fcdc8 None admin] RBAC: Authorizing `identity:create_grant(project_id=48926fa411ae43869359fdc0695cd3a8, user_id=d91da95dd8674152b0b2f5f61a71da6b, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:04.591472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ce3fc-cb6f-4882-8584-0e0a837fcdc8 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:04.611144 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 71/140] 10.222.0.81 () {58 vars in 1439 bytes} [Tue Jun 7 14:07:04 2022] PUT /identity/v3/projects/48926fa411ae43869359fdc0695cd3a8/users/d91da95dd8674152b0b2f5f61a71da6b/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:07:04.612003 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ce3fc-cb6f-4882-8584-0e0a837fcdc8 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:04.613638 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 70/141] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:07:04.670961 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e7eed-e64d-4eda-9524-461b18ca1559 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:04.723735 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:04.731788 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ce3fc-cb6f-4882-8584-0e0a837fcdc8 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:04.808349 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b139b-fa0d-4f5b-8ec5-12605d52f27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:04.809201 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b139b-fa0d-4f5b-8ec5-12605d52f27e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:04.809496 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b139b-fa0d-4f5b-8ec5-12605d52f27e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:04.809882 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b139b-fa0d-4f5b-8ec5-12605d52f27e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:04.829219 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b139b-fa0d-4f5b-8ec5-12605d52f27e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:04.840897 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-bd6b139b-fa0d-4f5b-8ec5-12605d52f27e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:07:04.840897 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:04.840897 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:04.840897 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:04.840897 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:04.840897 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:04.840897 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:04.840897 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:04.840897 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:04.840897 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:04.840897 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:04.840897 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:04.840897 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:04.840897 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:04.840897 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:04.840897 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:04.841928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:04.841928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:04.841928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:04.841928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:04.841928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:04.841928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:04.841928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:04.841928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:04.841928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:04.841928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:04.841928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:04.841928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:04.841928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:04.841928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:04.841928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:04.841928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:04.842706 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:04.842706 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:04.842706 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:04.842706 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:04.842706 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:04.842706 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:04.842706 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:04.842706 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:04.842706 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:04.842706 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:04.842706 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:07:04.842706 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:04.842706 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 72/142] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:04.846182 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e7eed-e64d-4eda-9524-461b18ca1559 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:04.853604 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b139b-fa0d-4f5b-8ec5-12605d52f27e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:04.889457 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db23359-75d2-438f-bb75-582f0f2a484a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:04.890283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db23359-75d2-438f-bb75-582f0f2a484a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:04.890574 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db23359-75d2-438f-bb75-582f0f2a484a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:04.890870 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db23359-75d2-438f-bb75-582f0f2a484a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:04.892142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db23359-75d2-438f-bb75-582f0f2a484a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:04.893627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db23359-75d2-438f-bb75-582f0f2a484a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:04.895621 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 71/143] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:07:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:04.901024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db23359-75d2-438f-bb75-582f0f2a484a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:04.999666 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:05.077377 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b139b-fa0d-4f5b-8ec5-12605d52f27e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:05.100770 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db23359-75d2-438f-bb75-582f0f2a484a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:05.141268 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a2e5b-fc8b-44f2-8a31-6cde8dc028de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:05.142373 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a2e5b-fc8b-44f2-8a31-6cde8dc028de None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:05.142858 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a2e5b-fc8b-44f2-8a31-6cde8dc028de None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:05.143385 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a2e5b-fc8b-44f2-8a31-6cde8dc028de None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:05.144378 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a2e5b-fc8b-44f2-8a31-6cde8dc028de None admin] RBAC: Adding query filter params (name=reader) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:05.144885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a2e5b-fc8b-44f2-8a31-6cde8dc028de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:05.146471 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a2e5b-fc8b-44f2-8a31-6cde8dc028de None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:05.159242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b82ec8f-de6f-4aa5-b4c1-c10b1fc66915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:05.159242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b82ec8f-de6f-4aa5-b4c1-c10b1fc66915 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:05.159242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b82ec8f-de6f-4aa5-b4c1-c10b1fc66915 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:05.159242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b82ec8f-de6f-4aa5-b4c1-c10b1fc66915 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:05.159946 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b82ec8f-de6f-4aa5-b4c1-c10b1fc66915 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:05.159946 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b82ec8f-de6f-4aa5-b4c1-c10b1fc66915 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:05.170303 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 73/144] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:07:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:05.176947 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a2e5b-fc8b-44f2-8a31-6cde8dc028de None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:05.210523 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 72/145] 10.222.0.81 () {60 vars in 1132 bytes} [Tue Jun 7 14:07:04 2022] POST /identity/v3/users => generated 283 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:07:05.211989 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:05.334060 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a2e5b-fc8b-44f2-8a31-6cde8dc028de None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:05.388865 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394e223-0314-4a05-992f-18c5e5eb2109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:05.390105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394e223-0314-4a05-992f-18c5e5eb2109 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:05.390573 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394e223-0314-4a05-992f-18c5e5eb2109 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:05.391028 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394e223-0314-4a05-992f-18c5e5eb2109 None admin] PATH_INFO: `/v3/users/b628faa9048645e18a23946e0e07b058` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:05.392463 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0394e223-0314-4a05-992f-18c5e5eb2109 None admin] RBAC: Authorizing `identity:get_user(user_id=b628faa9048645e18a23946e0e07b058)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:05.394132 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0394e223-0314-4a05-992f-18c5e5eb2109 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:05.396069 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 74/146] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:05 2022] GET /identity/v3/users/b628faa9048645e18a23946e0e07b058 => generated 289 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:05.401900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b82ec8f-de6f-4aa5-b4c1-c10b1fc66915 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:05.538703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b82ec8f-de6f-4aa5-b4c1-c10b1fc66915 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:05.540845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:05.579535 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e684e-493e-4acc-8852-4875533677ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:05.580418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e684e-493e-4acc-8852-4875533677ba None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:05.580700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e684e-493e-4acc-8852-4875533677ba None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:05.580974 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e684e-493e-4acc-8852-4875533677ba None admin] PATH_INFO: `/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:05.582309 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e684e-493e-4acc-8852-4875533677ba None admin] RBAC: Authorizing `identity:get_project(project_id=a85ae30482264d7ba1c84ab0ca46d4e0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:05.583864 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e684e-493e-4acc-8852-4875533677ba None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:05.586073 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 73/147] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:05 2022] GET /identity/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:05.591728 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394e223-0314-4a05-992f-18c5e5eb2109 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:05.719839 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:05.737758 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394e223-0314-4a05-992f-18c5e5eb2109 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:05.786952 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a5565-9516-4c0f-938d-6b86025a1571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:05.787891 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a5565-9516-4c0f-938d-6b86025a1571 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:05.788174 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a5565-9516-4c0f-938d-6b86025a1571 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:05.788536 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a5565-9516-4c0f-938d-6b86025a1571 None admin] PATH_INFO: `/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0/users/b628faa9048645e18a23946e0e07b058/roles/a1d21c4d26ef419298e001001f4655b7` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:05.792527 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a5565-9516-4c0f-938d-6b86025a1571 None admin] RBAC: Authorizing `identity:create_grant(project_id=a85ae30482264d7ba1c84ab0ca46d4e0, user_id=b628faa9048645e18a23946e0e07b058, role_id=a1d21c4d26ef419298e001001f4655b7)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:05.794010 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a5565-9516-4c0f-938d-6b86025a1571 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:05.815200 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 75/148] 10.222.0.81 () {58 vars in 1439 bytes} [Tue Jun 7 14:07:05 2022] PUT /identity/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0/users/b628faa9048645e18a23946e0e07b058/roles/a1d21c4d26ef419298e001001f4655b7 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:07:06.930412 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7db84-cdd6-47b0-a573-9f65ce1dc338 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:06.931741 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7db84-cdd6-47b0-a573-9f65ce1dc338 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:06.932231 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7db84-cdd6-47b0-a573-9f65ce1dc338 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:06.933833 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 74/149] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:06.939424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c1340-3303-402c-9e42-d5b8731bb07b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:06.940275 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c1340-3303-402c-9e42-d5b8731bb07b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:06.940748 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c1340-3303-402c-9e42-d5b8731bb07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:06.964206 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-417c1340-3303-402c-9e42-d5b8731bb07b None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:06.965793 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-417c1340-3303-402c-9e42-d5b8731bb07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:07.050274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 76/150] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:07.093399 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7db84-cdd6-47b0-a573-9f65ce1dc338 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:07.228808 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7db84-cdd6-47b0-a573-9f65ce1dc338 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:07.274114 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22156458-203d-48d3-9715-8433657902b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:07.275892 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22156458-203d-48d3-9715-8433657902b9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:07.276391 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22156458-203d-48d3-9715-8433657902b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:07.276857 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22156458-203d-48d3-9715-8433657902b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:07.291339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22156458-203d-48d3-9715-8433657902b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:07.300615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-22156458-203d-48d3-9715-8433657902b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:07:07.300615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:07.300615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:07.300615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:07.300615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:07.300615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:07.300615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:07.300615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:07.300615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:07.300615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:07.300615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:07.300615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:07.300615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:07.300615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:07.300615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:07.300615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:07.301563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:07.301563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:07.301563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:07.301563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:07.301563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:07.301563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:07.301563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:07.301563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:07.301563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:07.301563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:07.301563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:07.301563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:07.301563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:07.301563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:07.301563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:07.301563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:07.302329 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:07.302329 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:07.302329 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:07.302329 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:07.302329 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:07.302329 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:07.302329 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:07.302329 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:07.302329 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:07.302329 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:07.302329 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:07:07.302329 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:07.304773 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 75/151] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:07:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:07.311402 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c1340-3303-402c-9e42-d5b8731bb07b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:07.363900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca669823-c7bb-4d85-816c-65a9a46460ce None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:07.364620 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca669823-c7bb-4d85-816c-65a9a46460ce None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:07.365119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca669823-c7bb-4d85-816c-65a9a46460ce None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:07.366728 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 76/152] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:07.378066 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142caca-900f-4ac1-8311-3d2e18c34ca3 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:07.378631 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142caca-900f-4ac1-8311-3d2e18c34ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:07.379121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142caca-900f-4ac1-8311-3d2e18c34ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:07.403692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b142caca-900f-4ac1-8311-3d2e18c34ca3 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:07.405393 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b142caca-900f-4ac1-8311-3d2e18c34ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:07.437168 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:07.455567 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:07.514025 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 77/153] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:07:07.539037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c1340-3303-402c-9e42-d5b8731bb07b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:07.582162 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b142caca-900f-4ac1-8311-3d2e18c34ca3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:07.615956 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3460b5-5f5d-47ec-bf34-916e9eb83557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:07.616651 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3460b5-5f5d-47ec-bf34-916e9eb83557 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:07.616937 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3460b5-5f5d-47ec-bf34-916e9eb83557 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:07.617215 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3460b5-5f5d-47ec-bf34-916e9eb83557 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:07.617984 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3460b5-5f5d-47ec-bf34-916e9eb83557 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:07.618290 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3460b5-5f5d-47ec-bf34-916e9eb83557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:07.620554 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3460b5-5f5d-47ec-bf34-916e9eb83557 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:07.634739 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 77/154] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:07.640670 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3460b5-5f5d-47ec-bf34-916e9eb83557 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:07.722817 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b142caca-900f-4ac1-8311-3d2e18c34ca3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:07.767518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56741a1-c831-48ca-9bd3-1fe6f8ad8439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:07.768271 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56741a1-c831-48ca-9bd3-1fe6f8ad8439 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:07.768556 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56741a1-c831-48ca-9bd3-1fe6f8ad8439 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:07.768828 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56741a1-c831-48ca-9bd3-1fe6f8ad8439 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:07.789929 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56741a1-c831-48ca-9bd3-1fe6f8ad8439 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:07.798396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-e56741a1-c831-48ca-9bd3-1fe6f8ad8439 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:07.798396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:07.798396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:07.798396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:07.798396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:07.798396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:07.798396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:07.798396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:07.798396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:07.798396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:07.798396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:07.798396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:07.798396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:07.798396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:07.798396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:07.798396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:07.799343 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:07.799343 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:07.799343 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:07.799343 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:07.799343 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:07.799343 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:07.799343 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:07.799343 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:07.799343 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:07.799343 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:07.799343 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:07.799343 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:07.799343 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:07.799343 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:07.799343 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:07.799343 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:07.800306 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:07.800306 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:07.800306 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:07.800306 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:07.800306 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:07.800306 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:07.800306 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:07.800306 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:07.800306 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:07.800306 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:07.800306 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:07.800306 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:07.808253 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 78/155] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:07.809733 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c1e5f-f40a-4e48-a723-6c5a06f0fd82 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:07.810029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c1e5f-f40a-4e48-a723-6c5a06f0fd82 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:07.810308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c1e5f-f40a-4e48-a723-6c5a06f0fd82 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:07.811963 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 79/156] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:07.817436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31837dd-4022-4525-b4f0-79e2b4aa3b7c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:07.817725 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31837dd-4022-4525-b4f0-79e2b4aa3b7c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:07.818002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31837dd-4022-4525-b4f0-79e2b4aa3b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:07.846564 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c31837dd-4022-4525-b4f0-79e2b4aa3b7c None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:07.847789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c31837dd-4022-4525-b4f0-79e2b4aa3b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:07.868694 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:07.887940 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3460b5-5f5d-47ec-bf34-916e9eb83557 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:07.951968 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589638f-014a-4081-9ce9-0a399ed2614b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:07.952862 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589638f-014a-4081-9ce9-0a399ed2614b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:07.953148 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589638f-014a-4081-9ce9-0a399ed2614b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:07.953422 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589638f-014a-4081-9ce9-0a399ed2614b None admin] PATH_INFO: `/v3/users/2604a49f226a4bdf958c4047caf193e8` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:07.954715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1589638f-014a-4081-9ce9-0a399ed2614b None admin] RBAC: Authorizing `identity:get_user(user_id=2604a49f226a4bdf958c4047caf193e8)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:07.956197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1589638f-014a-4081-9ce9-0a399ed2614b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:07.958166 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 78/157] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:07 2022] GET /identity/v3/users/2604a49f226a4bdf958c4047caf193e8 => generated 247 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:07.959030 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589638f-014a-4081-9ce9-0a399ed2614b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:07.970460 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 80/158] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:07:07.971464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31837dd-4022-4525-b4f0-79e2b4aa3b7c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:07.990495 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:08.137036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589638f-014a-4081-9ce9-0a399ed2614b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:08.145252 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31837dd-4022-4525-b4f0-79e2b4aa3b7c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:08.178330 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bec232-3f96-4c04-8a8b-c8e16caba5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:08.179412 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bec232-3f96-4c04-8a8b-c8e16caba5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:08.179896 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bec232-3f96-4c04-8a8b-c8e16caba5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:08.180367 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bec232-3f96-4c04-8a8b-c8e16caba5d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:08.181862 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bec232-3f96-4c04-8a8b-c8e16caba5d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:08.183994 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bec232-3f96-4c04-8a8b-c8e16caba5d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:08.185614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bec232-3f96-4c04-8a8b-c8e16caba5d1 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:08.205953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 79/159] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:07:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:08.208055 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bec232-3f96-4c04-8a8b-c8e16caba5d1 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:08.243133 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc49c06-1894-42bd-92a0-d9eb94c0b0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:08.244430 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc49c06-1894-42bd-92a0-d9eb94c0b0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:08.244922 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc49c06-1894-42bd-92a0-d9eb94c0b0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:08.245389 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc49c06-1894-42bd-92a0-d9eb94c0b0d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:08.304743 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:08.355602 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:08.410796 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bec232-3f96-4c04-8a8b-c8e16caba5d1 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:08.444322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc49c06-1894-42bd-92a0-d9eb94c0b0d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:08.446057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc49c06-1894-42bd-92a0-d9eb94c0b0d2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:08.471010 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e215b7-3a35-4c36-9d00-97b157a6cc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:08.471867 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e215b7-3a35-4c36-9d00-97b157a6cc5a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:08.472152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e215b7-3a35-4c36-9d00-97b157a6cc5a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:08.472429 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e215b7-3a35-4c36-9d00-97b157a6cc5a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:08.496491 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e215b7-3a35-4c36-9d00-97b157a6cc5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:08.513298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-25e215b7-3a35-4c36-9d00-97b157a6cc5a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:07:08.513298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:08.513298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:08.513298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:08.513298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:08.513298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:08.513298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:08.513298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:08.513298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:08.513298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:08.513298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:08.513298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:08.513298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:08.513298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:08.513298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:08.513298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:08.514395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:08.514395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:08.514395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:08.514395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:08.514395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:08.514395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:08.514395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:08.514395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:08.514395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:08.514395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:08.514395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:08.514395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:08.514395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:08.514395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:08.514395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:08.514395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:08.515171 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:08.515171 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:08.515171 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:08.515171 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:08.515171 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:08.515171 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:08.515171 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:08.515171 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:08.515171 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:08.515171 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:08.515171 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:07:08.515171 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:08.515171 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 80/160] 10.222.0.81 () {56 vars in 1114 bytes} [Tue Jun 7 14:07:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:08.515839 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775bb1d-ec74-4f9e-bb6d-c14a2dfde4c1 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:08.516388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775bb1d-ec74-4f9e-bb6d-c14a2dfde4c1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:08.516388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775bb1d-ec74-4f9e-bb6d-c14a2dfde4c1 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:08.517640 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 81/161] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:08.518336 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775bb1d-ec74-4f9e-bb6d-c14a2dfde4c1 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:08.580244 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 81/162] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:07:08.581702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50fa1f-5e96-4f49-95b6-3796899826af None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:08.581997 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50fa1f-5e96-4f49-95b6-3796899826af None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:08.582274 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50fa1f-5e96-4f49-95b6-3796899826af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:08.617777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0b50fa1f-5e96-4f49-95b6-3796899826af None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:08.619999 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0b50fa1f-5e96-4f49-95b6-3796899826af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:08.699381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:08.723929 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775bb1d-ec74-4f9e-bb6d-c14a2dfde4c1 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:08.736177 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 82/163] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:08.737120 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50fa1f-5e96-4f49-95b6-3796899826af None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:08.782858 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:08.806202 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eaf080-1641-41fe-9793-a330d4f4c51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:08.807086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eaf080-1641-41fe-9793-a330d4f4c51d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:08.807402 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eaf080-1641-41fe-9793-a330d4f4c51d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:08.807682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eaf080-1641-41fe-9793-a330d4f4c51d None admin] PATH_INFO: `/v3/users/d91da95dd8674152b0b2f5f61a71da6b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:08.811950 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eaf080-1641-41fe-9793-a330d4f4c51d None admin] RBAC: Authorizing `identity:get_user(user_id=d91da95dd8674152b0b2f5f61a71da6b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:08.813429 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eaf080-1641-41fe-9793-a330d4f4c51d None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:08.815383 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 82/164] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:08 2022] GET /identity/v3/users/d91da95dd8674152b0b2f5f61a71da6b => generated 275 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:08.816268 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eaf080-1641-41fe-9793-a330d4f4c51d None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:08.884965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50fa1f-5e96-4f49-95b6-3796899826af None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:08.939866 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a3992-e59d-479a-a33e-1768a65bbfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:08.940661 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a3992-e59d-479a-a33e-1768a65bbfd2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:08.940917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a3992-e59d-479a-a33e-1768a65bbfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:08.941190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a3992-e59d-479a-a33e-1768a65bbfd2 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:08.941950 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a3992-e59d-479a-a33e-1768a65bbfd2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:08.942255 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a3992-e59d-479a-a33e-1768a65bbfd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:08.943672 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a3992-e59d-479a-a33e-1768a65bbfd2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:08.957822 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 83/165] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:07:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:08.958515 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a3992-e59d-479a-a33e-1768a65bbfd2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:09.007263 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eaf080-1641-41fe-9793-a330d4f4c51d None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:09.026798 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:09.084092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7976eae7-f8d0-4d15-b2fb-49ed35efd36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:09.084984 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976eae7-f8d0-4d15-b2fb-49ed35efd36e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:09.085265 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976eae7-f8d0-4d15-b2fb-49ed35efd36e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:09.085540 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976eae7-f8d0-4d15-b2fb-49ed35efd36e None admin] PATH_INFO: `/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:09.088506 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7976eae7-f8d0-4d15-b2fb-49ed35efd36e None admin] RBAC: Authorizing `identity:get_project(project_id=a85ae30482264d7ba1c84ab0ca46d4e0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:09.089956 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7976eae7-f8d0-4d15-b2fb-49ed35efd36e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:09.096397 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 83/166] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:08 2022] GET /identity/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0 => generated 298 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:09.097027 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7976eae7-f8d0-4d15-b2fb-49ed35efd36e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:09.159585 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a3992-e59d-479a-a33e-1768a65bbfd2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:09.214946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:09.223470 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56bab0c-a43c-4d5e-b02b-8572fd55543b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:09.224289 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bab0c-a43c-4d5e-b02b-8572fd55543b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:09.224572 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bab0c-a43c-4d5e-b02b-8572fd55543b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:09.224847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bab0c-a43c-4d5e-b02b-8572fd55543b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:09.250698 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56bab0c-a43c-4d5e-b02b-8572fd55543b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:09.257487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7976eae7-f8d0-4d15-b2fb-49ed35efd36e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:09.258409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-c56bab0c-a43c-4d5e-b02b-8572fd55543b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:09.258409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:09.258409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:09.258409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:09.258409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:09.258409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:09.258409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:09.258409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:09.258409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:09.258409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:09.258409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:09.258409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:09.258409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:09.258409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:09.258409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:09.258409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:09.260036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:09.260036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:09.260036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:09.260036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:09.260036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:09.260036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:09.260036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:09.260036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:09.260036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:09.260036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:09.260036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:09.260036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:09.260036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:09.260036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:09.260036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:09.260036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:09.261087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:09.261087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:09.261087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:09.261087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:09.261087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:09.261087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:09.261087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:09.261087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:09.261087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:09.261087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:09.261087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:09.261087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:09.268925 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 84/167] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:09.270235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56bab0c-a43c-4d5e-b02b-8572fd55543b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:09.298064 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aaea08-7232-4e06-b876-5ed60178297d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:09.298728 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aaea08-7232-4e06-b876-5ed60178297d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:09.299013 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aaea08-7232-4e06-b876-5ed60178297d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:09.299322 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aaea08-7232-4e06-b876-5ed60178297d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:09.372923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aaea08-7232-4e06-b876-5ed60178297d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:09.374668 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aaea08-7232-4e06-b876-5ed60178297d None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:09.495992 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56bab0c-a43c-4d5e-b02b-8572fd55543b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:09.497702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:09.499611 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:09.554167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 84/168] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:07:09.560205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aaea08-7232-4e06-b876-5ed60178297d None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:09.560473 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f65c7-4693-4e44-afdc-a93d260144ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:09.561386 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f65c7-4693-4e44-afdc-a93d260144ab None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:09.561670 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f65c7-4693-4e44-afdc-a93d260144ab None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:09.561946 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f65c7-4693-4e44-afdc-a93d260144ab None admin] PATH_INFO: `/v3/users/d91da95dd8674152b0b2f5f61a71da6b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:09.567311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f65c7-4693-4e44-afdc-a93d260144ab None admin] RBAC: Authorizing `identity:get_user(user_id=d91da95dd8674152b0b2f5f61a71da6b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:09.568835 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f65c7-4693-4e44-afdc-a93d260144ab None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:09.570503 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 85/169] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:09 2022] GET /identity/v3/users/d91da95dd8674152b0b2f5f61a71da6b => generated 275 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:09.572358 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f65c7-4693-4e44-afdc-a93d260144ab None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:09.668572 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aaea08-7232-4e06-b876-5ed60178297d None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:09.706965 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e2c32-61f7-464c-8dda-16e8ddd98276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:09.708039 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e2c32-61f7-464c-8dda-16e8ddd98276 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:09.708494 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e2c32-61f7-464c-8dda-16e8ddd98276 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:09.708984 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e2c32-61f7-464c-8dda-16e8ddd98276 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:09.709976 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e2c32-61f7-464c-8dda-16e8ddd98276 None admin] RBAC: Adding query filter params (role.id=82bbe3651b9b4f378fd150d93357b2b1, scope.project.id=a85ae30482264d7ba1c84ab0ca46d4e0, user.id=2604a49f226a4bdf958c4047caf193e8) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:09.710445 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e2c32-61f7-464c-8dda-16e8ddd98276 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:09.712487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e2c32-61f7-464c-8dda-16e8ddd98276 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:09.719244 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 85/170] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:07:09 2022] GET /identity/v3/role_assignments?user.id=2604a49f226a4bdf958c4047caf193e8&scope.project.id=a85ae30482264d7ba1c84ab0ca46d4e0&role.id=82bbe3651b9b4f378fd150d93357b2b1 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:09.727447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7ef83-1a88-431d-902c-69186a573135 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:09.727660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f65c7-4693-4e44-afdc-a93d260144ab None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:09.728207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7ef83-1a88-431d-902c-69186a573135 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:09.728635 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7ef83-1a88-431d-902c-69186a573135 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:09.730505 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 86/171] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:09 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:09.731439 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7ef83-1a88-431d-902c-69186a573135 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:09.789988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:09.792756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39aa73e-89f5-4bc3-9b7d-fddb734b3101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:09.793659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39aa73e-89f5-4bc3-9b7d-fddb734b3101 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:09.794025 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39aa73e-89f5-4bc3-9b7d-fddb734b3101 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:09.794345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39aa73e-89f5-4bc3-9b7d-fddb734b3101 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:09.795385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39aa73e-89f5-4bc3-9b7d-fddb734b3101 None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:09.795835 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39aa73e-89f5-4bc3-9b7d-fddb734b3101 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:09.797442 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39aa73e-89f5-4bc3-9b7d-fddb734b3101 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:09.825458 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 86/172] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:07:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:09.826211 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39aa73e-89f5-4bc3-9b7d-fddb734b3101 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:09.907846 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7ef83-1a88-431d-902c-69186a573135 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:09.928184 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:09.988068 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f783881-16ff-4a4e-a315-98097cd82287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:09.988969 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f783881-16ff-4a4e-a315-98097cd82287 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:09.989255 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f783881-16ff-4a4e-a315-98097cd82287 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:09.989528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f783881-16ff-4a4e-a315-98097cd82287 None admin] PATH_INFO: `/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:09.996138 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f783881-16ff-4a4e-a315-98097cd82287 None admin] RBAC: Authorizing `identity:get_project(project_id=a85ae30482264d7ba1c84ab0ca46d4e0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:09.998183 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f783881-16ff-4a4e-a315-98097cd82287 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:10.000209 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 87/173] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:09 2022] GET /identity/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0 => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:10.001578 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074d32e-9bf6-421f-b7b4-a13ece2ef8e8 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:10.002010 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074d32e-9bf6-421f-b7b4-a13ece2ef8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:10.002429 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074d32e-9bf6-421f-b7b4-a13ece2ef8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:10.036280 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-c074d32e-9bf6-421f-b7b4-a13ece2ef8e8 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:10.043324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39aa73e-89f5-4bc3-9b7d-fddb734b3101 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:10.043815 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c074d32e-9bf6-421f-b7b4-a13ece2ef8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:10.108046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438020d-e3f8-4eff-9b79-63f5a1389081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:10.108967 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438020d-e3f8-4eff-9b79-63f5a1389081 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:10.109429 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438020d-e3f8-4eff-9b79-63f5a1389081 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:10.109840 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438020d-e3f8-4eff-9b79-63f5a1389081 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:10.160858 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 88/174] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:10.161730 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c074d32e-9bf6-421f-b7b4-a13ece2ef8e8 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:10.191663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:10.271988 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6438020d-e3f8-4eff-9b79-63f5a1389081 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:10.273820 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6438020d-e3f8-4eff-9b79-63f5a1389081 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:10.284656 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c074d32e-9bf6-421f-b7b4-a13ece2ef8e8 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:10.320245 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:10.339623 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6ca35-c1bb-41f2-bbc2-bca05d0bd9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:10.341099 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6ca35-c1bb-41f2-bbc2-bca05d0bd9dd None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:10.342130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6ca35-c1bb-41f2-bbc2-bca05d0bd9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:10.342572 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6ca35-c1bb-41f2-bbc2-bca05d0bd9dd None admin] PATH_INFO: `/v3/users/d91da95dd8674152b0b2f5f61a71da6b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:10.344483 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c6ca35-c1bb-41f2-bbc2-bca05d0bd9dd None admin] RBAC: Authorizing `identity:get_user(user_id=d91da95dd8674152b0b2f5f61a71da6b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:10.346913 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c6ca35-c1bb-41f2-bbc2-bca05d0bd9dd None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:10.348863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 89/175] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:10 2022] GET /identity/v3/users/d91da95dd8674152b0b2f5f61a71da6b => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:10.350450 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6ca35-c1bb-41f2-bbc2-bca05d0bd9dd None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:10.412113 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 87/176] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:07:10.413296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438020d-e3f8-4eff-9b79-63f5a1389081 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:10.479626 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:10.485657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6ca35-c1bb-41f2-bbc2-bca05d0bd9dd None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:10.529661 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a21bb1-0367-48c0-b607-e80e4a5fac20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:10.530660 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a21bb1-0367-48c0-b607-e80e4a5fac20 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:10.531238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a21bb1-0367-48c0-b607-e80e4a5fac20 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:10.531663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a21bb1-0367-48c0-b607-e80e4a5fac20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:10.532720 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a21bb1-0367-48c0-b607-e80e4a5fac20 None admin] RBAC: Adding query filter params (role.id=ca91f64e7a46431c8aa4285c4620f216, scope.project.id=a85ae30482264d7ba1c84ab0ca46d4e0, user.id=d91da95dd8674152b0b2f5f61a71da6b) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:10.533265 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a21bb1-0367-48c0-b607-e80e4a5fac20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:10.534962 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a21bb1-0367-48c0-b607-e80e4a5fac20 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:10.540513 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438020d-e3f8-4eff-9b79-63f5a1389081 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:10.541647 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 90/177] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:07:10 2022] GET /identity/v3/role_assignments?user.id=d91da95dd8674152b0b2f5f61a71da6b&scope.project.id=a85ae30482264d7ba1c84ab0ca46d4e0&role.id=ca91f64e7a46431c8aa4285c4620f216 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:10.543416 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f30d6-547a-49e3-9d77-24a95227dfab None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:10.543934 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f30d6-547a-49e3-9d77-24a95227dfab None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:10.546341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f30d6-547a-49e3-9d77-24a95227dfab None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:10.548534 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 91/178] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:10.549624 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f30d6-547a-49e3-9d77-24a95227dfab None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:10.603289 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:10.616566 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff70b15-47ca-43f0-b97c-d4dcf0556546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:10.617763 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff70b15-47ca-43f0-b97c-d4dcf0556546 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:10.618837 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff70b15-47ca-43f0-b97c-d4dcf0556546 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:10.619379 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff70b15-47ca-43f0-b97c-d4dcf0556546 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:10.646915 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff70b15-47ca-43f0-b97c-d4dcf0556546 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:10.661068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-eff70b15-47ca-43f0-b97c-d4dcf0556546 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:07:10.661068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:10.661068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:10.661068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:10.661068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:10.661068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:10.661068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:10.661068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:10.661068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:10.661068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:10.661068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:10.661068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:10.661068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:10.661068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:10.661068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:10.661068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:10.662080 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:10.662080 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:10.662080 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:10.662080 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:10.662080 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:10.662080 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:10.662080 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:10.662080 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:10.662080 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:10.662080 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:10.662080 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:10.662080 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:10.662080 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:10.662080 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:10.662080 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:10.662080 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:10.662908 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:10.662908 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:10.662908 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:10.662908 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:10.662908 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:10.662908 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:10.662908 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:10.662908 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:10.662908 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:10.662908 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:10.662908 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:07:10.662908 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:10.664865 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 88/179] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:10.665652 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff70b15-47ca-43f0-b97c-d4dcf0556546 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:10.744419 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f30d6-547a-49e3-9d77-24a95227dfab None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:10.784117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b9d82b-ddea-4e98-b344-3fc618191b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:10.784777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9d82b-ddea-4e98-b344-3fc618191b41 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:10.785059 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9d82b-ddea-4e98-b344-3fc618191b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:10.785331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9d82b-ddea-4e98-b344-3fc618191b41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:10.787494 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff70b15-47ca-43f0-b97c-d4dcf0556546 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:10.817762 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:10.829221 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ec613-1f4e-4395-ba36-4362df7d0c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:10.830095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ec613-1f4e-4395-ba36-4362df7d0c0c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:10.830376 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ec613-1f4e-4395-ba36-4362df7d0c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:10.830648 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ec613-1f4e-4395-ba36-4362df7d0c0c None admin] PATH_INFO: `/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:10.831879 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ec613-1f4e-4395-ba36-4362df7d0c0c None admin] RBAC: Authorizing `identity:get_project(project_id=a85ae30482264d7ba1c84ab0ca46d4e0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:10.833335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ec613-1f4e-4395-ba36-4362df7d0c0c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:10.835297 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 89/180] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:10 2022] GET /identity/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:10.836223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea888e-59c4-49e7-85b0-8a4c3cde3d88 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:10.836508 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea888e-59c4-49e7-85b0-8a4c3cde3d88 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:10.836785 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea888e-59c4-49e7-85b0-8a4c3cde3d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:10.853867 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f4ea888e-59c4-49e7-85b0-8a4c3cde3d88 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:10.855018 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f4ea888e-59c4-49e7-85b0-8a4c3cde3d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:10.864652 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b9d82b-ddea-4e98-b344-3fc618191b41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:10.866119 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b9d82b-ddea-4e98-b344-3fc618191b41 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:10.909684 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:10.921925 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 90/181] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:10.924541 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fe1b3-da56-4d78-97f4-e4e1fabee64d None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:10.924820 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fe1b3-da56-4d78-97f4-e4e1fabee64d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:10.925093 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fe1b3-da56-4d78-97f4-e4e1fabee64d None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:10.926372 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 91/182] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:10.926372 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876fe1b3-da56-4d78-97f4-e4e1fabee64d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:10.972091 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 92/183] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:07:10.973417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed45e8-fb27-4938-beb7-d7c5d0c3b01d None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:10.973701 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed45e8-fb27-4938-beb7-d7c5d0c3b01d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:10.973971 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed45e8-fb27-4938-beb7-d7c5d0c3b01d None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:10.975248 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 93/184] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:10.975939 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed45e8-fb27-4938-beb7-d7c5d0c3b01d None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:11.035309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:11.042821 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876fe1b3-da56-4d78-97f4-e4e1fabee64d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:11.077890 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54fa04c-87bd-4082-85bf-2da9e85a1417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:11.078547 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed45e8-fb27-4938-beb7-d7c5d0c3b01d None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:11.078896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fa04c-87bd-4082-85bf-2da9e85a1417 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:11.078896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fa04c-87bd-4082-85bf-2da9e85a1417 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:11.079093 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fa04c-87bd-4082-85bf-2da9e85a1417 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:11.079885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54fa04c-87bd-4082-85bf-2da9e85a1417 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:11.080190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54fa04c-87bd-4082-85bf-2da9e85a1417 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:11.081568 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54fa04c-87bd-4082-85bf-2da9e85a1417 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:11.090240 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 92/185] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:07:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:11.092504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380052a9-a505-43dd-a650-3f43cee2d989 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:11.092786 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380052a9-a505-43dd-a650-3f43cee2d989 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:11.093064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380052a9-a505-43dd-a650-3f43cee2d989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:11.109656 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-380052a9-a505-43dd-a650-3f43cee2d989 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:11.110776 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-380052a9-a505-43dd-a650-3f43cee2d989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:11.115350 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9817814-ecbb-4a4d-9a80-b3f53243b5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:11.116012 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9817814-ecbb-4a4d-9a80-b3f53243b5d3 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:11.116294 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9817814-ecbb-4a4d-9a80-b3f53243b5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:11.116568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9817814-ecbb-4a4d-9a80-b3f53243b5d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:11.117396 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9817814-ecbb-4a4d-9a80-b3f53243b5d3 None admin] RBAC: Adding query filter params (role.id=6d02db1fe03b447c9e519f4f2aa16574, scope.project.id=a85ae30482264d7ba1c84ab0ca46d4e0, user.id=d91da95dd8674152b0b2f5f61a71da6b) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:11.117709 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9817814-ecbb-4a4d-9a80-b3f53243b5d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:11.119403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9817814-ecbb-4a4d-9a80-b3f53243b5d3 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:11.125447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 94/186] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:07:10 2022] GET /identity/v3/role_assignments?user.id=d91da95dd8674152b0b2f5f61a71da6b&scope.project.id=a85ae30482264d7ba1c84ab0ca46d4e0&role.id=6d02db1fe03b447c9e519f4f2aa16574 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:11.126224 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9817814-ecbb-4a4d-9a80-b3f53243b5d3 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:11.171039 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:11.196505 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 93/187] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:11.205026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380052a9-a505-43dd-a650-3f43cee2d989 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:11.243308 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9817814-ecbb-4a4d-9a80-b3f53243b5d3 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:11.269245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:11.284162 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943ec55-aa40-42a2-803f-b3612a322b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:11.284975 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943ec55-aa40-42a2-803f-b3612a322b73 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:11.285263 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943ec55-aa40-42a2-803f-b3612a322b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:11.285551 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943ec55-aa40-42a2-803f-b3612a322b73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:11.297777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5943ec55-aa40-42a2-803f-b3612a322b73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:11.305707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-5943ec55-aa40-42a2-803f-b3612a322b73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:07:11.305707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:11.305707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:11.305707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:11.305707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:11.305707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:11.305707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:11.305707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:11.305707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:11.305707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:11.305707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:11.305707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:11.305707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:11.305707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:11.305707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:11.305707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:11.306788 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:11.306788 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:11.306788 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:11.306788 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:11.306788 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:11.306788 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:11.306788 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:11.306788 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:11.306788 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:11.306788 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:11.306788 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:11.306788 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:11.306788 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:11.306788 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:11.306788 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:11.306788 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:11.307626 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:11.307626 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:11.307626 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:11.307626 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:11.307626 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:11.307626 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:11.307626 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:11.307626 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:11.307626 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:11.307626 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:11.307626 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:07:11.307626 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:11.312221 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 95/188] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:07:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:11.313452 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7326919-d9f7-468d-a3b0-f0a221deea7b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:11.313746 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7326919-d9f7-468d-a3b0-f0a221deea7b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:11.314028 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7326919-d9f7-468d-a3b0-f0a221deea7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:11.329308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380052a9-a505-43dd-a650-3f43cee2d989 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:11.332267 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-c7326919-d9f7-468d-a3b0-f0a221deea7b None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:11.333699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c7326919-d9f7-468d-a3b0-f0a221deea7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:11.369143 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9034dd-1419-4a89-968e-ff11b536176f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:11.369940 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9034dd-1419-4a89-968e-ff11b536176f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:11.370549 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9034dd-1419-4a89-968e-ff11b536176f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:11.370549 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9034dd-1419-4a89-968e-ff11b536176f None admin] PATH_INFO: `/v3/projects/48926fa411ae43869359fdc0695cd3a8` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:11.371744 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9034dd-1419-4a89-968e-ff11b536176f None admin] RBAC: Authorizing `identity:get_project(project_id=48926fa411ae43869359fdc0695cd3a8)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:11.373481 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9034dd-1419-4a89-968e-ff11b536176f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:11.375196 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 94/189] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:11 2022] GET /identity/v3/projects/48926fa411ae43869359fdc0695cd3a8 => generated 312 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:11.375975 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9034dd-1419-4a89-968e-ff11b536176f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:11.410753 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 96/190] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:11.411637 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7326919-d9f7-468d-a3b0-f0a221deea7b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:11.437530 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:11.488208 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9034dd-1419-4a89-968e-ff11b536176f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:11.512331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7326919-d9f7-468d-a3b0-f0a221deea7b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:11.523349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e965c-eade-4662-b727-fe4fa1e02f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:11.524222 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e965c-eade-4662-b727-fe4fa1e02f04 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:11.524592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e965c-eade-4662-b727-fe4fa1e02f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:11.525470 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e965c-eade-4662-b727-fe4fa1e02f04 None admin] PATH_INFO: `/v3/users/b628faa9048645e18a23946e0e07b058` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:11.525750 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:11.526599 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e965c-eade-4662-b727-fe4fa1e02f04 None admin] RBAC: Authorizing `identity:get_user(user_id=b628faa9048645e18a23946e0e07b058)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:11.528081 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e965c-eade-4662-b727-fe4fa1e02f04 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:11.529731 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 95/191] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:11 2022] GET /identity/v3/users/b628faa9048645e18a23946e0e07b058 => generated 289 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:11.530501 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e965c-eade-4662-b727-fe4fa1e02f04 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:11.547571 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c80c07-7372-4fa1-b43a-565bf306812c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:11.548396 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c80c07-7372-4fa1-b43a-565bf306812c None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:11.548682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c80c07-7372-4fa1-b43a-565bf306812c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:11.548961 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c80c07-7372-4fa1-b43a-565bf306812c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:11.559508 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c80c07-7372-4fa1-b43a-565bf306812c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:11.567639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-b1c80c07-7372-4fa1-b43a-565bf306812c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:07:11.567639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:11.567639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:11.567639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:11.567639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:11.567639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:11.567639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:11.567639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:11.567639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:11.567639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:11.567639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:11.567639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:11.567639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:11.567639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:11.567639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:11.567639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:11.568740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:11.568740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:11.568740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:11.568740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:11.568740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:11.568740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:11.568740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:11.568740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:11.568740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:11.568740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:11.568740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:11.568740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:11.568740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:11.568740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:11.568740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:11.568740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:11.569546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:11.569546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:11.569546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:11.569546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:11.569546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:11.569546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:11.569546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:11.569546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:11.569546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:11.569546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:11.569546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:07:11.569546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:11.569546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 97/192] 10.222.0.81 () {56 vars in 1114 bytes} [Tue Jun 7 14:07:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:11.570220 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c80c07-7372-4fa1-b43a-565bf306812c None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:11.635209 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e965c-eade-4662-b727-fe4fa1e02f04 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:11.651954 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:11.670820 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91815d8-c63b-46cc-ad21-f19ba4f4e358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:11.671049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c80c07-7372-4fa1-b43a-565bf306812c None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:11.671531 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91815d8-c63b-46cc-ad21-f19ba4f4e358 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:11.671802 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91815d8-c63b-46cc-ad21-f19ba4f4e358 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:11.672074 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91815d8-c63b-46cc-ad21-f19ba4f4e358 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:11.672982 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91815d8-c63b-46cc-ad21-f19ba4f4e358 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:11.673283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91815d8-c63b-46cc-ad21-f19ba4f4e358 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:11.674644 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91815d8-c63b-46cc-ad21-f19ba4f4e358 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:11.683065 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 96/193] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:11.683065 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91815d8-c63b-46cc-ad21-f19ba4f4e358 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:11.708525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71baf55b-93f8-432f-a15b-a3d2684a383b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:11.709177 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71baf55b-93f8-432f-a15b-a3d2684a383b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:11.709458 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71baf55b-93f8-432f-a15b-a3d2684a383b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:11.709729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71baf55b-93f8-432f-a15b-a3d2684a383b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:11.710555 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71baf55b-93f8-432f-a15b-a3d2684a383b None admin] RBAC: Adding query filter params (role.id=ca91f64e7a46431c8aa4285c4620f216, scope.project.id=48926fa411ae43869359fdc0695cd3a8, user.id=d91da95dd8674152b0b2f5f61a71da6b) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:11.710859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71baf55b-93f8-432f-a15b-a3d2684a383b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:11.712637 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71baf55b-93f8-432f-a15b-a3d2684a383b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:11.718313 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 98/194] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:07:11 2022] GET /identity/v3/role_assignments?user.id=d91da95dd8674152b0b2f5f61a71da6b&scope.project.id=48926fa411ae43869359fdc0695cd3a8&role.id=ca91f64e7a46431c8aa4285c4620f216 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:11.719153 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71baf55b-93f8-432f-a15b-a3d2684a383b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:11.760617 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:11.800336 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91815d8-c63b-46cc-ad21-f19ba4f4e358 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:11.835753 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71baf55b-93f8-432f-a15b-a3d2684a383b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:11.838696 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5377f8-2815-4d9c-aa29-c52fee159bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:11.839550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5377f8-2815-4d9c-aa29-c52fee159bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:11.839834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5377f8-2815-4d9c-aa29-c52fee159bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:11.840117 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5377f8-2815-4d9c-aa29-c52fee159bd1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:11.841367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5377f8-2815-4d9c-aa29-c52fee159bd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:11.842846 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5377f8-2815-4d9c-aa29-c52fee159bd1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:11.844756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 97/195] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:07:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:11.845361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5377f8-2815-4d9c-aa29-c52fee159bd1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:11.883368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce304598-8344-437c-a009-0456874f8573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:11.884960 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce304598-8344-437c-a009-0456874f8573 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:11.885457 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce304598-8344-437c-a009-0456874f8573 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:11.885677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce304598-8344-437c-a009-0456874f8573 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:11.902787 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:11.966956 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5377f8-2815-4d9c-aa29-c52fee159bd1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:11.973709 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce304598-8344-437c-a009-0456874f8573 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:11.975185 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce304598-8344-437c-a009-0456874f8573 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:12.005224 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103ddfe-71a0-4a36-8eb3-2c348159daa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:12.005927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103ddfe-71a0-4a36-8eb3-2c348159daa6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:12.006207 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103ddfe-71a0-4a36-8eb3-2c348159daa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:12.006485 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103ddfe-71a0-4a36-8eb3-2c348159daa6 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:12.007251 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103ddfe-71a0-4a36-8eb3-2c348159daa6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:12.007565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103ddfe-71a0-4a36-8eb3-2c348159daa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:12.009645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103ddfe-71a0-4a36-8eb3-2c348159daa6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:12.013548 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:12.017711 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 98/196] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:07:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:12.018505 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103ddfe-71a0-4a36-8eb3-2c348159daa6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:12.090210 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 99/197] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:07:12.090210 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce304598-8344-437c-a009-0456874f8573 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:12.121206 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103ddfe-71a0-4a36-8eb3-2c348159daa6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:12.132355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:12.156883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02704f33-2013-4a07-900a-b01d2b7e168e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:12.157691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02704f33-2013-4a07-900a-b01d2b7e168e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:12.157970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02704f33-2013-4a07-900a-b01d2b7e168e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:12.158247 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02704f33-2013-4a07-900a-b01d2b7e168e None admin] PATH_INFO: `/v3/users/ac6a7b74505c418a884fc268487b7b8d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:12.173144 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02704f33-2013-4a07-900a-b01d2b7e168e None admin] RBAC: Authorizing `identity:get_user(user_id=ac6a7b74505c418a884fc268487b7b8d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:12.174662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02704f33-2013-4a07-900a-b01d2b7e168e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:12.176421 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 99/198] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:12 2022] GET /identity/v3/users/ac6a7b74505c418a884fc268487b7b8d => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:12.181734 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02704f33-2013-4a07-900a-b01d2b7e168e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:12.192518 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce304598-8344-437c-a009-0456874f8573 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:12.226993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf29180-8f9a-4ca3-b068-5bfc7720fa5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:12.227723 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf29180-8f9a-4ca3-b068-5bfc7720fa5b None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:12.228009 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf29180-8f9a-4ca3-b068-5bfc7720fa5b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:12.228293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf29180-8f9a-4ca3-b068-5bfc7720fa5b None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:12.228999 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf29180-8f9a-4ca3-b068-5bfc7720fa5b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:12.230434 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf29180-8f9a-4ca3-b068-5bfc7720fa5b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:12.237877 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:12.252149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 100/199] 10.222.0.81 () {60 vars in 1132 bytes} [Tue Jun 7 14:07:12 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:07:12.263565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf29180-8f9a-4ca3-b068-5bfc7720fa5b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:12.286491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02704f33-2013-4a07-900a-b01d2b7e168e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:12.327129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301cf9dc-7204-42b0-91cb-68e30f192362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:12.328066 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301cf9dc-7204-42b0-91cb-68e30f192362 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:12.328351 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301cf9dc-7204-42b0-91cb-68e30f192362 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:12.328629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301cf9dc-7204-42b0-91cb-68e30f192362 None admin] PATH_INFO: `/v3/users/ac6a7b74505c418a884fc268487b7b8d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:12.329905 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301cf9dc-7204-42b0-91cb-68e30f192362 None admin] RBAC: Authorizing `identity:get_user(user_id=ac6a7b74505c418a884fc268487b7b8d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:12.331431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301cf9dc-7204-42b0-91cb-68e30f192362 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:12.333145 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 100/200] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:12 2022] GET /identity/v3/users/ac6a7b74505c418a884fc268487b7b8d => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:12.333933 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301cf9dc-7204-42b0-91cb-68e30f192362 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:12.374813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:12.379300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf29180-8f9a-4ca3-b068-5bfc7720fa5b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:12.421885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db529d-7067-412e-82f9-437bef100a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:12.422667 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db529d-7067-412e-82f9-437bef100a1e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:12.422950 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db529d-7067-412e-82f9-437bef100a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:12.423256 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db529d-7067-412e-82f9-437bef100a1e None admin] PATH_INFO: `/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:12.424568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0db529d-7067-412e-82f9-437bef100a1e None admin] RBAC: Authorizing `identity:get_project(project_id=a85ae30482264d7ba1c84ab0ca46d4e0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:12.425939 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0db529d-7067-412e-82f9-437bef100a1e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:12.433249 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 101/201] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:12 2022] GET /identity/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:12.433249 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db529d-7067-412e-82f9-437bef100a1e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:12.450595 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301cf9dc-7204-42b0-91cb-68e30f192362 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:12.496864 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a75e29-a34c-48d1-9d37-fb5965f910a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:12.497788 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a75e29-a34c-48d1-9d37-fb5965f910a0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:12.498066 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a75e29-a34c-48d1-9d37-fb5965f910a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:12.498337 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a75e29-a34c-48d1-9d37-fb5965f910a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:12.506886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:12.558971 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db529d-7067-412e-82f9-437bef100a1e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:12.596750 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a75e29-a34c-48d1-9d37-fb5965f910a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:12.598399 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a75e29-a34c-48d1-9d37-fb5965f910a0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:12.606616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1c8cf-bdd3-4b95-a955-fc599ac735b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:12.607317 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1c8cf-bdd3-4b95-a955-fc599ac735b8 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:12.607595 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1c8cf-bdd3-4b95-a955-fc599ac735b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:12.607875 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1c8cf-bdd3-4b95-a955-fc599ac735b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:12.646692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:12.695121 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b1c8cf-bdd3-4b95-a955-fc599ac735b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:12.697904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b1c8cf-bdd3-4b95-a955-fc599ac735b8 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:12.741793 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 101/202] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:07:12.742959 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a75e29-a34c-48d1-9d37-fb5965f910a0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:12.781494 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:12.828411 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 102/203] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:07:12.829694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1c8cf-bdd3-4b95-a955-fc599ac735b8 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:12.856202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a75e29-a34c-48d1-9d37-fb5965f910a0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:12.895024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e136af82-0c58-44f2-852f-f584923acfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:12.895791 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136af82-0c58-44f2-852f-f584923acfb1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:12.896006 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136af82-0c58-44f2-852f-f584923acfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:12.896308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136af82-0c58-44f2-852f-f584923acfb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:12.897117 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e136af82-0c58-44f2-852f-f584923acfb1 None admin] RBAC: Adding query filter params (role.id=a1d21c4d26ef419298e001001f4655b7, scope.project.id=a85ae30482264d7ba1c84ab0ca46d4e0, user.id=b628faa9048645e18a23946e0e07b058) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:12.897505 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e136af82-0c58-44f2-852f-f584923acfb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:12.898861 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e136af82-0c58-44f2-852f-f584923acfb1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:12.902020 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:12.911630 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 102/204] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:07:12 2022] GET /identity/v3/role_assignments?user.id=b628faa9048645e18a23946e0e07b058&scope.project.id=a85ae30482264d7ba1c84ab0ca46d4e0&role.id=a1d21c4d26ef419298e001001f4655b7 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:12.912590 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e136af82-0c58-44f2-852f-f584923acfb1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:12.948308 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1c8cf-bdd3-4b95-a955-fc599ac735b8 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:12.989505 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ca79f-89c8-4d0b-b925-accd16758e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:12.990768 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ca79f-89c8-4d0b-b925-accd16758e1f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:12.991287 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ca79f-89c8-4d0b-b925-accd16758e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:12.991904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ca79f-89c8-4d0b-b925-accd16758e1f None admin] PATH_INFO: `/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:12.993622 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ca79f-89c8-4d0b-b925-accd16758e1f None admin] RBAC: Authorizing `identity:get_project(project_id=c05a3987827c46a69dd1ab7b7f4b913b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:12.995640 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ca79f-89c8-4d0b-b925-accd16758e1f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:12.998217 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 103/205] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:12 2022] GET /identity/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:13.004883 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ca79f-89c8-4d0b-b925-accd16758e1f None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:13.039461 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:13.126328 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ca79f-89c8-4d0b-b925-accd16758e1f None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:13.169385 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c814ff-c404-448d-a35f-95e0c0c5820b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:13.170350 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c814ff-c404-448d-a35f-95e0c0c5820b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:13.170825 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c814ff-c404-448d-a35f-95e0c0c5820b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:13.171315 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c814ff-c404-448d-a35f-95e0c0c5820b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:13.172355 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c814ff-c404-448d-a35f-95e0c0c5820b None admin] RBAC: Adding query filter params (role.id=82bbe3651b9b4f378fd150d93357b2b1, scope.project.id=c05a3987827c46a69dd1ab7b7f4b913b, user.id=ac6a7b74505c418a884fc268487b7b8d) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:13.172860 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c814ff-c404-448d-a35f-95e0c0c5820b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:13.173403 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e136af82-0c58-44f2-852f-f584923acfb1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:13.175166 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c814ff-c404-448d-a35f-95e0c0c5820b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:13.182167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 104/206] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:07:13 2022] GET /identity/v3/role_assignments?user.id=ac6a7b74505c418a884fc268487b7b8d&scope.project.id=c05a3987827c46a69dd1ab7b7f4b913b&role.id=82bbe3651b9b4f378fd150d93357b2b1 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:13.216415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe9e99-91f0-4644-ae8f-3e6f733f930f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:13.217315 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe9e99-91f0-4644-ae8f-3e6f733f930f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:13.217596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe9e99-91f0-4644-ae8f-3e6f733f930f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:13.217869 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe9e99-91f0-4644-ae8f-3e6f733f930f None admin] PATH_INFO: `/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:13.219169 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe9e99-91f0-4644-ae8f-3e6f733f930f None admin] RBAC: Authorizing `identity:get_project(project_id=c05a3987827c46a69dd1ab7b7f4b913b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:13.220791 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe9e99-91f0-4644-ae8f-3e6f733f930f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:13.222685 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 103/207] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:12 2022] GET /identity/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b => generated 302 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:13.229599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c814ff-c404-448d-a35f-95e0c0c5820b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:13.337940 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:13.348557 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c814ff-c404-448d-a35f-95e0c0c5820b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:13.397036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ba20c-86c8-4313-9be5-f4cf23f33837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:13.397723 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ba20c-86c8-4313-9be5-f4cf23f33837 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:13.398002 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ba20c-86c8-4313-9be5-f4cf23f33837 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:13.398276 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ba20c-86c8-4313-9be5-f4cf23f33837 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:13.399118 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ba20c-86c8-4313-9be5-f4cf23f33837 None admin] RBAC: Adding query filter params (role.id=6d02db1fe03b447c9e519f4f2aa16574, scope.project.id=c05a3987827c46a69dd1ab7b7f4b913b, user.id=ac6a7b74505c418a884fc268487b7b8d) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:13.399987 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ba20c-86c8-4313-9be5-f4cf23f33837 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:13.401921 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ba20c-86c8-4313-9be5-f4cf23f33837 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:13.408586 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 105/208] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:07:13 2022] GET /identity/v3/role_assignments?user.id=ac6a7b74505c418a884fc268487b7b8d&scope.project.id=c05a3987827c46a69dd1ab7b7f4b913b&role.id=6d02db1fe03b447c9e519f4f2aa16574 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:14.773728 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25fb45-3b9b-4776-a51f-3c4a871431b7 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:14.775544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25fb45-3b9b-4776-a51f-3c4a871431b7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:14.776030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25fb45-3b9b-4776-a51f-3c4a871431b7 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:14.777895 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 104/209] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:14.783376 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9bc8de-c2df-40c8-9a8b-3ce5e9fa62ee None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:14.784181 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9bc8de-c2df-40c8-9a8b-3ce5e9fa62ee None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:14.784657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9bc8de-c2df-40c8-9a8b-3ce5e9fa62ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:14.801948 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-8b9bc8de-c2df-40c8-9a8b-3ce5e9fa62ee None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:14.803493 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8b9bc8de-c2df-40c8-9a8b-3ce5e9fa62ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:14.826090 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbc96b-04db-47ef-a1d1-c008cdbbf316 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:14.826697 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbc96b-04db-47ef-a1d1-c008cdbbf316 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:14.827178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbc96b-04db-47ef-a1d1-c008cdbbf316 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:14.829075 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 105/210] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:07:14.834284 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc187250-4f33-4bd7-bc2a-4a1401dcf801 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:14.834788 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc187250-4f33-4bd7-bc2a-4a1401dcf801 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:14.835294 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc187250-4f33-4bd7-bc2a-4a1401dcf801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:14.854704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-cc187250-4f33-4bd7-bc2a-4a1401dcf801 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:14.856366 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cc187250-4f33-4bd7-bc2a-4a1401dcf801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:14.884796 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 106/211] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:14.909519 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9bc8de-c2df-40c8-9a8b-3ce5e9fa62ee None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:14.926277 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:14.933860 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 106/212] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:07:14.957332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc187250-4f33-4bd7-bc2a-4a1401dcf801 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:15.007593 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:15.026819 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9bc8de-c2df-40c8-9a8b-3ce5e9fa62ee None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:15.062798 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc187250-4f33-4bd7-bc2a-4a1401dcf801 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:15.070794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb2154-84cd-4d91-8736-2a166a0f7314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:15.072354 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb2154-84cd-4d91-8736-2a166a0f7314 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:15.072640 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb2154-84cd-4d91-8736-2a166a0f7314 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:15.072920 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb2154-84cd-4d91-8736-2a166a0f7314 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:15.074122 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb2154-84cd-4d91-8736-2a166a0f7314 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:15.076392 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb2154-84cd-4d91-8736-2a166a0f7314 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:15.078504 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 107/213] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:07:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:15.084219 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb2154-84cd-4d91-8736-2a166a0f7314 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:15.108836 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08aad5-f660-42e3-8b44-c57db305e2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:15.109996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08aad5-f660-42e3-8b44-c57db305e2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:15.110347 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08aad5-f660-42e3-8b44-c57db305e2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:15.110684 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08aad5-f660-42e3-8b44-c57db305e2a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:15.128130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08aad5-f660-42e3-8b44-c57db305e2a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:15.138913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-9c08aad5-f660-42e3-8b44-c57db305e2a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:15.138913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:15.138913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:15.138913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:15.138913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:15.138913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:15.138913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:15.138913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:15.138913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:15.138913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:15.138913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:15.138913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:15.138913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:15.138913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:15.138913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:15.138913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:15.139844 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:15.139844 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:15.139844 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:15.139844 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:15.139844 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:15.139844 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:15.139844 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:15.139844 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:15.139844 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:15.139844 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:15.139844 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:15.139844 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:15.139844 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:15.139844 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:15.139844 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:15.139844 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:15.140647 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:15.140647 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:15.140647 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:15.140647 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:15.140647 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:15.140647 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:15.140647 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:15.140647 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:15.140647 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:15.140647 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:15.140647 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:15.140647 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:15.141252 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 107/214] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:07:15.146932 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08aad5-f660-42e3-8b44-c57db305e2a8 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:15.210231 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb2154-84cd-4d91-8736-2a166a0f7314 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:15.228696 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:15.254191 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d83bb-cc16-4329-8f31-4147e6423e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:15.255367 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d83bb-cc16-4329-8f31-4147e6423e7c None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:15.255868 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d83bb-cc16-4329-8f31-4147e6423e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:15.256438 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d83bb-cc16-4329-8f31-4147e6423e7c None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:15.257368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39d83bb-cc16-4329-8f31-4147e6423e7c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:15.259846 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39d83bb-cc16-4329-8f31-4147e6423e7c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:15.284043 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08aad5-f660-42e3-8b44-c57db305e2a8 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:15.287356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 108/215] 10.222.0.81 () {60 vars in 1132 bytes} [Tue Jun 7 14:07:15 2022] POST /identity/v3/users => generated 297 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:07:15.330181 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a772b3b-4c73-4dd9-8966-e71bd25bd97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:15.331275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a772b3b-4c73-4dd9-8966-e71bd25bd97a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:15.331697 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a772b3b-4c73-4dd9-8966-e71bd25bd97a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:15.332107 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a772b3b-4c73-4dd9-8966-e71bd25bd97a None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:15.333019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a772b3b-4c73-4dd9-8966-e71bd25bd97a None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:15.333466 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a772b3b-4c73-4dd9-8966-e71bd25bd97a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:15.335029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a772b3b-4c73-4dd9-8966-e71bd25bd97a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:15.346198 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 108/216] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:07:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:15.352036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d83bb-cc16-4329-8f31-4147e6423e7c None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:15.368400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:15.478716 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d83bb-cc16-4329-8f31-4147e6423e7c None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:15.521769 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ecaf11-6ffd-43aa-96e3-5c7dabd497b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:15.522671 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecaf11-6ffd-43aa-96e3-5c7dabd497b5 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:15.522957 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecaf11-6ffd-43aa-96e3-5c7dabd497b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:15.523244 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecaf11-6ffd-43aa-96e3-5c7dabd497b5 None admin] PATH_INFO: `/v3/users/0e68e1214136444590de9239e1a0fc84` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:15.539888 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ecaf11-6ffd-43aa-96e3-5c7dabd497b5 None admin] RBAC: Authorizing `identity:get_user(user_id=0e68e1214136444590de9239e1a0fc84)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:15.541499 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ecaf11-6ffd-43aa-96e3-5c7dabd497b5 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:15.543595 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 109/217] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:15 2022] GET /identity/v3/users/0e68e1214136444590de9239e1a0fc84 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:15.548895 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a772b3b-4c73-4dd9-8966-e71bd25bd97a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:15.560766 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6e1ae-d8ee-4b24-ab35-b8937751fe0e None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:15.560766 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6e1ae-d8ee-4b24-ab35-b8937751fe0e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:15.561040 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6e1ae-d8ee-4b24-ab35-b8937751fe0e None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:15.563545 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 110/218] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:07:15.570324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055a188-ce6d-4485-a215-d96801e22e66 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:15.570612 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055a188-ce6d-4485-a215-d96801e22e66 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:15.570889 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055a188-ce6d-4485-a215-d96801e22e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:15.593348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-7055a188-ce6d-4485-a215-d96801e22e66 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:15.594574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7055a188-ce6d-4485-a215-d96801e22e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:15.678687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:15.679732 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a772b3b-4c73-4dd9-8966-e71bd25bd97a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:15.691820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 111/219] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:15.705308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:15.721469 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055a188-ce6d-4485-a215-d96801e22e66 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:15.722496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021dcee0-9f3e-4871-874d-2d48efae5948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:15.723163 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dcee0-9f3e-4871-874d-2d48efae5948 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:15.723535 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dcee0-9f3e-4871-874d-2d48efae5948 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:15.723816 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dcee0-9f3e-4871-874d-2d48efae5948 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:15.808470 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021dcee0-9f3e-4871-874d-2d48efae5948 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:15.810596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021dcee0-9f3e-4871-874d-2d48efae5948 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:15.847413 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055a188-ce6d-4485-a215-d96801e22e66 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:15.894746 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b7849-fdbb-40f0-be1b-9d46dae95964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:15.895576 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b7849-fdbb-40f0-be1b-9d46dae95964 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:15.895862 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b7849-fdbb-40f0-be1b-9d46dae95964 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:15.896672 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b7849-fdbb-40f0-be1b-9d46dae95964 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:15.910401 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b7849-fdbb-40f0-be1b-9d46dae95964 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:15.922369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-7a2b7849-fdbb-40f0-be1b-9d46dae95964 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:07:15.922369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:15.922369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:15.922369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:15.922369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:15.922369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:15.922369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:15.922369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:15.922369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:15.922369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:15.922369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:15.922369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:15.922369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:15.922369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:15.922369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:15.922369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:15.923516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:15.923516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:15.923516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:15.923516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:15.923516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:15.923516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:15.923516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:15.923516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:15.923516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:15.923516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:15.923516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:15.923516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:15.923516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:15.923516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:15.923516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:15.923516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:15.924369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:15.924369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:15.924369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:15.924369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:15.924369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:15.924369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:15.924369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:15.924369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:15.924369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:15.924369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:15.924369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:07:15.924369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:15.924369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 112/220] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:07:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:07:15.925401 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14241b8-4798-4d5b-b40b-413bc4e7383c None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:15.925505 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 109/221] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 14:07:15.925590 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14241b8-4798-4d5b-b40b-413bc4e7383c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:15.925729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14241b8-4798-4d5b-b40b-413bc4e7383c None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:15.926989 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 113/222] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:15.930628 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021dcee0-9f3e-4871-874d-2d48efae5948 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:15.933047 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903de53-813e-4894-94e4-96c5bbd43767 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:15.933337 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903de53-813e-4894-94e4-96c5bbd43767 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:15.933925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903de53-813e-4894-94e4-96c5bbd43767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:15.955807 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:15.961613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-c903de53-813e-4894-94e4-96c5bbd43767 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:15.963789 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c903de53-813e-4894-94e4-96c5bbd43767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:16.033032 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:16.049234 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 114/223] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:16.050595 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903de53-813e-4894-94e4-96c5bbd43767 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:16.051649 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021dcee0-9f3e-4871-874d-2d48efae5948 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:16.094697 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa539f-bafa-477f-aee5-c7e40dd7b66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:16.095933 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa539f-bafa-477f-aee5-c7e40dd7b66a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:16.096417 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa539f-bafa-477f-aee5-c7e40dd7b66a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:16.096893 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa539f-bafa-477f-aee5-c7e40dd7b66a None admin] PATH_INFO: `/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:16.098340 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa539f-bafa-477f-aee5-c7e40dd7b66a None admin] RBAC: Authorizing `identity:get_project(project_id=c05a3987827c46a69dd1ab7b7f4b913b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:16.100165 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa539f-bafa-477f-aee5-c7e40dd7b66a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:16.102307 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 110/224] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:15 2022] GET /identity/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:16.104122 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa539f-bafa-477f-aee5-c7e40dd7b66a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:16.184396 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903de53-813e-4894-94e4-96c5bbd43767 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:16.217065 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:16.221472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6015a4b-19c0-4ca9-a8d9-fa8c6e7bc1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:16.222386 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6015a4b-19c0-4ca9-a8d9-fa8c6e7bc1dd None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:16.222568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6015a4b-19c0-4ca9-a8d9-fa8c6e7bc1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:16.222802 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6015a4b-19c0-4ca9-a8d9-fa8c6e7bc1dd None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:16.223576 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6015a4b-19c0-4ca9-a8d9-fa8c6e7bc1dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:16.223881 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6015a4b-19c0-4ca9-a8d9-fa8c6e7bc1dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:16.225253 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6015a4b-19c0-4ca9-a8d9-fa8c6e7bc1dd None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:16.233620 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa539f-bafa-477f-aee5-c7e40dd7b66a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:16.238765 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 115/225] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:16.239592 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6015a4b-19c0-4ca9-a8d9-fa8c6e7bc1dd None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:16.274682 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acfe60-e225-44da-b289-6a107eeab9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:16.275954 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acfe60-e225-44da-b289-6a107eeab9b9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:16.276559 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acfe60-e225-44da-b289-6a107eeab9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:16.277166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acfe60-e225-44da-b289-6a107eeab9b9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:16.288729 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2acfe60-e225-44da-b289-6a107eeab9b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:16.292490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:16.300493 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-b2acfe60-e225-44da-b289-6a107eeab9b9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:07:16.300493 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:16.300493 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:16.300493 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:16.300493 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:16.300493 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:16.300493 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:16.300493 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:16.300493 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:16.300493 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:16.300493 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:16.300493 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:16.300493 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:16.300493 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:16.300493 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:16.300493 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:16.301416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:16.301416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:16.301416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:16.301416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:16.301416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:16.301416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:16.301416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:16.301416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:16.301416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:16.301416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:16.301416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:16.301416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:16.301416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:16.301416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:16.301416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:16.301416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:16.302205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:16.302205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:16.302205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:16.302205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:16.302205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:16.302205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:16.302205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:16.302205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:16.302205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:16.302205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:16.302205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:07:16.302205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:16.304834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 111/226] 10.222.0.81 () {56 vars in 1114 bytes} [Tue Jun 7 14:07:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:16.306731 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acfe60-e225-44da-b289-6a107eeab9b9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:16.351350 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6015a4b-19c0-4ca9-a8d9-fa8c6e7bc1dd None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:16.386916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87bb56-d34b-435e-9b38-e68cdb47ffd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:16.387685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87bb56-d34b-435e-9b38-e68cdb47ffd1 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:16.387972 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87bb56-d34b-435e-9b38-e68cdb47ffd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:16.388245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87bb56-d34b-435e-9b38-e68cdb47ffd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:16.389084 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87bb56-d34b-435e-9b38-e68cdb47ffd1 None admin] RBAC: Adding query filter params (role.id=ca91f64e7a46431c8aa4285c4620f216, scope.project.id=c05a3987827c46a69dd1ab7b7f4b913b, user.id=0e68e1214136444590de9239e1a0fc84) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:16.389394 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87bb56-d34b-435e-9b38-e68cdb47ffd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:16.390838 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87bb56-d34b-435e-9b38-e68cdb47ffd1 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:16.397098 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 116/227] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:07:16 2022] GET /identity/v3/role_assignments?user.id=0e68e1214136444590de9239e1a0fc84&scope.project.id=c05a3987827c46a69dd1ab7b7f4b913b&role.id=ca91f64e7a46431c8aa4285c4620f216 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:16.397917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87bb56-d34b-435e-9b38-e68cdb47ffd1 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:16.437331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acfe60-e225-44da-b289-6a107eeab9b9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:16.479036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61981fd-f495-43f1-af6b-be617e3cdd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:16.480436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61981fd-f495-43f1-af6b-be617e3cdd28 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:16.480925 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61981fd-f495-43f1-af6b-be617e3cdd28 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:16.481464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61981fd-f495-43f1-af6b-be617e3cdd28 None admin] PATH_INFO: `/v3/users/ac6a7b74505c418a884fc268487b7b8d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:16.483318 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61981fd-f495-43f1-af6b-be617e3cdd28 None admin] RBAC: Authorizing `identity:get_user(user_id=ac6a7b74505c418a884fc268487b7b8d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:16.485089 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61981fd-f495-43f1-af6b-be617e3cdd28 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:16.487463 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 112/228] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:16 2022] GET /identity/v3/users/ac6a7b74505c418a884fc268487b7b8d => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:16.492579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61981fd-f495-43f1-af6b-be617e3cdd28 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:16.520831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:16.522178 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87bb56-d34b-435e-9b38-e68cdb47ffd1 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:16.544187 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:16.592753 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14f3e5-1805-4ae0-a413-fc6343977a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:16.595015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14f3e5-1805-4ae0-a413-fc6343977a61 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:16.596256 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14f3e5-1805-4ae0-a413-fc6343977a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:16.596689 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14f3e5-1805-4ae0-a413-fc6343977a61 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:16.598378 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14f3e5-1805-4ae0-a413-fc6343977a61 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:16.599034 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14f3e5-1805-4ae0-a413-fc6343977a61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:16.601818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14f3e5-1805-4ae0-a413-fc6343977a61 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:16.620822 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 117/229] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:07:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:16.627061 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14f3e5-1805-4ae0-a413-fc6343977a61 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:16.695271 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61981fd-f495-43f1-af6b-be617e3cdd28 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:16.733429 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6c1b4-533e-46c1-996d-e66f9da5778a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:16.734392 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6c1b4-533e-46c1-996d-e66f9da5778a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:16.734744 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6c1b4-533e-46c1-996d-e66f9da5778a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:16.735098 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6c1b4-533e-46c1-996d-e66f9da5778a None admin] PATH_INFO: `/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:16.736429 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6c1b4-533e-46c1-996d-e66f9da5778a None admin] RBAC: Authorizing `identity:get_project(project_id=c05a3987827c46a69dd1ab7b7f4b913b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:16.737959 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6c1b4-533e-46c1-996d-e66f9da5778a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:16.740114 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 113/230] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:16 2022] GET /identity/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:16.745772 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6c1b4-533e-46c1-996d-e66f9da5778a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:16.768082 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14f3e5-1805-4ae0-a413-fc6343977a61 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:16.844779 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fb0b5-8cb6-4f6b-aed5-bcb3d60de7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:16.845879 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fb0b5-8cb6-4f6b-aed5-bcb3d60de7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:16.846340 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fb0b5-8cb6-4f6b-aed5-bcb3d60de7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:16.846715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fb0b5-8cb6-4f6b-aed5-bcb3d60de7b7 None admin] PATH_INFO: `/v3/users/ac6a7b74505c418a884fc268487b7b8d` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:16.849589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:16.851407 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2fb0b5-8cb6-4f6b-aed5-bcb3d60de7b7 None admin] RBAC: Authorizing `identity:get_user(user_id=ac6a7b74505c418a884fc268487b7b8d)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:16.861060 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2fb0b5-8cb6-4f6b-aed5-bcb3d60de7b7 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:16.863357 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 118/231] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:16 2022] GET /identity/v3/users/ac6a7b74505c418a884fc268487b7b8d => generated 283 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:16.870147 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fb0b5-8cb6-4f6b-aed5-bcb3d60de7b7 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:16.871273 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6c1b4-533e-46c1-996d-e66f9da5778a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:16.913084 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123e168-1b8d-45d5-bad7-6ebc98df619e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:16.914012 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123e168-1b8d-45d5-bad7-6ebc98df619e None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:16.914299 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123e168-1b8d-45d5-bad7-6ebc98df619e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:16.915715 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123e168-1b8d-45d5-bad7-6ebc98df619e None admin] PATH_INFO: `/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b/users/ac6a7b74505c418a884fc268487b7b8d/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:16.917901 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3123e168-1b8d-45d5-bad7-6ebc98df619e None admin] RBAC: Authorizing `identity:create_grant(project_id=c05a3987827c46a69dd1ab7b7f4b913b, user_id=ac6a7b74505c418a884fc268487b7b8d, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:16.919798 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3123e168-1b8d-45d5-bad7-6ebc98df619e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:16.937399 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 114/232] 10.222.0.81 () {58 vars in 1439 bytes} [Tue Jun 7 14:07:16 2022] PUT /identity/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b/users/ac6a7b74505c418a884fc268487b7b8d/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:07:16.951770 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:17.018550 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fb0b5-8cb6-4f6b-aed5-bcb3d60de7b7 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:17.058193 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd6b50-3f79-4b01-a100-35c50dba9693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:17.059084 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd6b50-3f79-4b01-a100-35c50dba9693 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:17.059405 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd6b50-3f79-4b01-a100-35c50dba9693 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:17.059683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd6b50-3f79-4b01-a100-35c50dba9693 None admin] PATH_INFO: `/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:17.061022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd6b50-3f79-4b01-a100-35c50dba9693 None admin] RBAC: Authorizing `identity:get_project(project_id=c05a3987827c46a69dd1ab7b7f4b913b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:17.062545 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd6b50-3f79-4b01-a100-35c50dba9693 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:17.064975 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 119/233] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:16 2022] GET /identity/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:17.070458 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123e168-1b8d-45d5-bad7-6ebc98df619e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:17.191469 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123e168-1b8d-45d5-bad7-6ebc98df619e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:17.227011 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23407a-88da-4280-8188-04fd0580509e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:17.228039 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23407a-88da-4280-8188-04fd0580509e None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:17.228327 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23407a-88da-4280-8188-04fd0580509e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:17.228605 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23407a-88da-4280-8188-04fd0580509e None admin] PATH_INFO: `/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b/users/ac6a7b74505c418a884fc268487b7b8d/roles/6d02db1fe03b447c9e519f4f2aa16574` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:17.230830 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23407a-88da-4280-8188-04fd0580509e None admin] RBAC: Authorizing `identity:create_grant(project_id=c05a3987827c46a69dd1ab7b7f4b913b, user_id=ac6a7b74505c418a884fc268487b7b8d, role_id=6d02db1fe03b447c9e519f4f2aa16574)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:17.232318 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23407a-88da-4280-8188-04fd0580509e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:17.247300 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 115/234] 10.222.0.81 () {58 vars in 1439 bytes} [Tue Jun 7 14:07:17 2022] PUT /identity/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b/users/ac6a7b74505c418a884fc268487b7b8d/roles/6d02db1fe03b447c9e519f4f2aa16574 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:07:18.081445 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6b1a4-949b-4f3c-8f93-3d882d2db34e None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:18.081952 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6b1a4-949b-4f3c-8f93-3d882d2db34e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:18.082028 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6b1a4-949b-4f3c-8f93-3d882d2db34e None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:18.083407 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 120/235] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:18.088993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef8f69-51f1-45d4-8c7e-643af5df0dc0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:18.089393 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef8f69-51f1-45d4-8c7e-643af5df0dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:18.089572 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef8f69-51f1-45d4-8c7e-643af5df0dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:18.095760 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:18.117409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-dcef8f69-51f1-45d4-8c7e-643af5df0dc0 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:18.119908 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dcef8f69-51f1-45d4-8c7e-643af5df0dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:18.207169 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede20839-6871-4c3a-a41c-5976e2aaf272 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:18.207513 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede20839-6871-4c3a-a41c-5976e2aaf272 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:18.207756 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede20839-6871-4c3a-a41c-5976e2aaf272 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:18.209428 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 121/236] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:18.217212 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b4b68-0e00-405f-aa32-56b475457d37 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:18.217212 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b4b68-0e00-405f-aa32-56b475457d37 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:18.217758 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b4b68-0e00-405f-aa32-56b475457d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:18.224971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 116/237] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:18.241919 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a10b4b68-0e00-405f-aa32-56b475457d37 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:18.243323 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a10b4b68-0e00-405f-aa32-56b475457d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:18.314908 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef8f69-51f1-45d4-8c7e-643af5df0dc0 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:18.340713 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 122/238] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:07:18.377654 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b4b68-0e00-405f-aa32-56b475457d37 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:18.434063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef8f69-51f1-45d4-8c7e-643af5df0dc0 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:18.472928 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2652ac1-e12d-40c5-bdab-802cfceb365b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:18.473817 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2652ac1-e12d-40c5-bdab-802cfceb365b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:18.474109 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2652ac1-e12d-40c5-bdab-802cfceb365b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:18.474383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2652ac1-e12d-40c5-bdab-802cfceb365b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:18.481839 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:18.487306 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:18.505451 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2652ac1-e12d-40c5-bdab-802cfceb365b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:18.516408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-c2652ac1-e12d-40c5-bdab-802cfceb365b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:07:18.516408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:18.516408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:18.516408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:18.516408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:18.516408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:18.516408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:18.516408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:18.516408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:18.516408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:18.516408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:18.516408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:18.516408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:18.516408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:18.516408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:18.516408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:18.517296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:18.517296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:18.517296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:18.517296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:18.517296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:18.517296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:18.517296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:18.517296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:18.517296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:18.517296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:18.517296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:18.517296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:18.517296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:18.517296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:18.517296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:18.517296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:18.518083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:18.518083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:18.518083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:18.518083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:18.518083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:18.518083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:18.518083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:18.518083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:18.518083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:18.518083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:18.518083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:07:18.518083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:18.518083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 117/239] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:07:18.523887 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2652ac1-e12d-40c5-bdab-802cfceb365b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:18.589359 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b4b68-0e00-405f-aa32-56b475457d37 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:18.658572 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee9ed1-6559-484a-9811-70a74f94babd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:18.659385 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee9ed1-6559-484a-9811-70a74f94babd None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:18.659725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee9ed1-6559-484a-9811-70a74f94babd None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:18.660005 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee9ed1-6559-484a-9811-70a74f94babd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:18.661332 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee9ed1-6559-484a-9811-70a74f94babd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:18.664866 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee9ed1-6559-484a-9811-70a74f94babd None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:18.666830 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 123/240] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:07:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:18.672465 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee9ed1-6559-484a-9811-70a74f94babd None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:18.713914 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2652ac1-e12d-40c5-bdab-802cfceb365b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:18.756233 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58298f55-604a-4a23-933d-fd17aaa188a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:18.756918 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58298f55-604a-4a23-933d-fd17aaa188a5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:18.757203 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58298f55-604a-4a23-933d-fd17aaa188a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:18.757480 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58298f55-604a-4a23-933d-fd17aaa188a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:18.759446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58298f55-604a-4a23-933d-fd17aaa188a5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:18.759760 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58298f55-604a-4a23-933d-fd17aaa188a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:18.761224 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58298f55-604a-4a23-933d-fd17aaa188a5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:18.770976 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 118/241] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:07:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:18.778236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58298f55-604a-4a23-933d-fd17aaa188a5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:18.867290 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee9ed1-6559-484a-9811-70a74f94babd None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:18.884619 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:18.885260 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:18.934940 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700409d-fc94-45b7-9355-33d0b1879c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:18.937056 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700409d-fc94-45b7-9355-33d0b1879c3f None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:18.938280 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700409d-fc94-45b7-9355-33d0b1879c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:18.938730 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700409d-fc94-45b7-9355-33d0b1879c3f None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:18.939638 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2700409d-fc94-45b7-9355-33d0b1879c3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:18.943931 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2700409d-fc94-45b7-9355-33d0b1879c3f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:18.969459 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58298f55-604a-4a23-933d-fd17aaa188a5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:18.976507 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 124/242] 10.222.0.81 () {60 vars in 1132 bytes} [Tue Jun 7 14:07:18 2022] POST /identity/v3/users => generated 293 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:07:19.011285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086ff63-84c6-4746-88d4-f0929f8e5659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:19.012546 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086ff63-84c6-4746-88d4-f0929f8e5659 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:19.013004 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086ff63-84c6-4746-88d4-f0929f8e5659 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:19.013475 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086ff63-84c6-4746-88d4-f0929f8e5659 None admin] PATH_INFO: `/v3/users/da9e8f9fd6c54493bdca6f956c2eb20f` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:19.037753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5086ff63-84c6-4746-88d4-f0929f8e5659 None admin] RBAC: Authorizing `identity:get_user(user_id=da9e8f9fd6c54493bdca6f956c2eb20f)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:19.039633 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5086ff63-84c6-4746-88d4-f0929f8e5659 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:19.042313 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 119/243] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:18 2022] GET /identity/v3/users/da9e8f9fd6c54493bdca6f956c2eb20f => generated 297 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:19.048161 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700409d-fc94-45b7-9355-33d0b1879c3f None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:19.176585 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700409d-fc94-45b7-9355-33d0b1879c3f None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:19.213430 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92fb147-ff3b-4643-be7f-cd1cbe6a3c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:19.214150 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92fb147-ff3b-4643-be7f-cd1cbe6a3c31 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:19.214433 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92fb147-ff3b-4643-be7f-cd1cbe6a3c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:19.214708 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92fb147-ff3b-4643-be7f-cd1cbe6a3c31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:19.250989 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:19.288653 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92fb147-ff3b-4643-be7f-cd1cbe6a3c31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:19.290217 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92fb147-ff3b-4643-be7f-cd1cbe6a3c31 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:19.393218 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 125/244] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:07:19.398560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086ff63-84c6-4746-88d4-f0929f8e5659 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:19.480396 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e32b2-f18d-4a6a-a381-75ccd36febf2 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:19.480909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e32b2-f18d-4a6a-a381-75ccd36febf2 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:19.481327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e32b2-f18d-4a6a-a381-75ccd36febf2 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:19.482846 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 126/245] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:07:19.488784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e355d3d-5464-4594-8aa0-7f23dd2dd2ed None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:19.489239 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e355d3d-5464-4594-8aa0-7f23dd2dd2ed None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:19.489682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e355d3d-5464-4594-8aa0-7f23dd2dd2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:19.510731 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-0e355d3d-5464-4594-8aa0-7f23dd2dd2ed None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:19.512296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0e355d3d-5464-4594-8aa0-7f23dd2dd2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:19.515002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086ff63-84c6-4746-88d4-f0929f8e5659 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:19.550469 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:19.568781 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2ef3e-98d6-456f-954a-051b60ca9b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:19.569976 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2ef3e-98d6-456f-954a-051b60ca9b49 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:19.570263 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2ef3e-98d6-456f-954a-051b60ca9b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:19.570540 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2ef3e-98d6-456f-954a-051b60ca9b49 None admin] PATH_INFO: `/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:19.572301 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2ef3e-98d6-456f-954a-051b60ca9b49 None admin] RBAC: Authorizing `identity:get_project(project_id=c05a3987827c46a69dd1ab7b7f4b913b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:19.574452 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2ef3e-98d6-456f-954a-051b60ca9b49 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:19.576615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 120/246] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:19 2022] GET /identity/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:19.582556 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2ef3e-98d6-456f-954a-051b60ca9b49 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:19.585779 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:19.603585 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 127/247] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:19.627833 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e355d3d-5464-4594-8aa0-7f23dd2dd2ed None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:19.716771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2ef3e-98d6-456f-954a-051b60ca9b49 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:19.762431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93fa553-b0f9-4d33-b817-87c0649d370b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:19.763030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fa553-b0f9-4d33-b817-87c0649d370b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:19.763461 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fa553-b0f9-4d33-b817-87c0649d370b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:19.764040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fa553-b0f9-4d33-b817-87c0649d370b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:19.765303 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93fa553-b0f9-4d33-b817-87c0649d370b None admin] RBAC: Adding query filter params (role.id=a1d21c4d26ef419298e001001f4655b7, scope.project.id=c05a3987827c46a69dd1ab7b7f4b913b, user.id=da9e8f9fd6c54493bdca6f956c2eb20f) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:19.765626 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93fa553-b0f9-4d33-b817-87c0649d370b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:19.767079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93fa553-b0f9-4d33-b817-87c0649d370b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:19.776463 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 121/248] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:07:19 2022] GET /identity/v3/role_assignments?user.id=da9e8f9fd6c54493bdca6f956c2eb20f&scope.project.id=c05a3987827c46a69dd1ab7b7f4b913b&role.id=a1d21c4d26ef419298e001001f4655b7 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:19.789549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e355d3d-5464-4594-8aa0-7f23dd2dd2ed None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:19.820319 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf4106-4ae9-4cce-b0da-3d602c92b274 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:19.820602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf4106-4ae9-4cce-b0da-3d602c92b274 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:19.820883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf4106-4ae9-4cce-b0da-3d602c92b274 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:19.822624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 122/249] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:07:19.829043 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95543203-f7ac-4670-836a-d68ebee586b4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:19.829338 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95543203-f7ac-4670-836a-d68ebee586b4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:19.829773 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95543203-f7ac-4670-836a-d68ebee586b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:19.840859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f49a156-608e-4013-b52e-f2344496ba62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:19.841689 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f49a156-608e-4013-b52e-f2344496ba62 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:19.841972 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f49a156-608e-4013-b52e-f2344496ba62 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:19.842247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f49a156-608e-4013-b52e-f2344496ba62 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:19.853684 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-95543203-f7ac-4670-836a-d68ebee586b4 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:19.855044 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-95543203-f7ac-4670-836a-d68ebee586b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:19.857001 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f49a156-608e-4013-b52e-f2344496ba62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:19.868499 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-4f49a156-608e-4013-b52e-f2344496ba62 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:19.868499 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:19.868499 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:19.868499 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:19.868499 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:19.868499 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:19.868499 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:19.868499 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:19.868499 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:19.868499 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:19.868499 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:19.868499 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:19.868499 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:19.868499 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:19.868499 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:19.868499 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:19.869617 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:19.869617 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:19.869617 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:19.869617 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:19.869617 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:19.869617 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:19.869617 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:19.869617 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:19.869617 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:19.869617 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:19.869617 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:19.869617 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:19.869617 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:19.869617 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:19.869617 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:19.869617 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:19.870424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:19.870424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:19.870424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:19.870424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:19.870424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:19.870424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:19.870424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:19.870424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:19.870424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:19.870424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:19.870424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:19.870424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:19.871106 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 128/250] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:19.878575 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f49a156-608e-4013-b52e-f2344496ba62 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:19.913407 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:19.951374 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 123/251] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:07:19.958980 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:19.971825 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95543203-f7ac-4670-836a-d68ebee586b4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:20.018350 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f49a156-608e-4013-b52e-f2344496ba62 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:20.068262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cccfd84-4bed-4436-a27d-2b5401512d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:20.068945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cccfd84-4bed-4436-a27d-2b5401512d24 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:20.069390 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cccfd84-4bed-4436-a27d-2b5401512d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:20.069673 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cccfd84-4bed-4436-a27d-2b5401512d24 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:20.070970 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cccfd84-4bed-4436-a27d-2b5401512d24 None admin] RBAC: Adding query filter params (name=member) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:20.071597 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cccfd84-4bed-4436-a27d-2b5401512d24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:20.073294 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cccfd84-4bed-4436-a27d-2b5401512d24 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:20.084978 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 129/252] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:07:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:20.091188 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cccfd84-4bed-4436-a27d-2b5401512d24 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:20.091506 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95543203-f7ac-4670-836a-d68ebee586b4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:20.132116 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1621f9-1de3-4932-a3b5-1d0df8b08b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:20.133084 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1621f9-1de3-4932-a3b5-1d0df8b08b5b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:20.133452 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1621f9-1de3-4932-a3b5-1d0df8b08b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:20.133888 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1621f9-1de3-4932-a3b5-1d0df8b08b5b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:20.145835 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1621f9-1de3-4932-a3b5-1d0df8b08b5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:20.154989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-af1621f9-1de3-4932-a3b5-1d0df8b08b5b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:07:20.154989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:20.154989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:20.154989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:20.154989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:20.154989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:20.154989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:20.154989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:20.154989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:20.154989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:20.154989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:20.154989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:20.154989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:20.154989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:20.154989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:20.154989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:20.155953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:20.155953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:20.155953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:20.155953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:20.155953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:20.155953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:20.155953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:20.155953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:20.155953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:20.155953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:20.155953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:20.155953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:20.155953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:20.155953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:20.155953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:20.155953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:20.156734 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:20.156734 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:20.156734 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:20.156734 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:20.156734 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:20.156734 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:20.156734 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:20.156734 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:20.156734 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:20.156734 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:20.156734 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:07:20.156734 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:20.161106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 124/253] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:07:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:20.162345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1621f9-1de3-4932-a3b5-1d0df8b08b5b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:20.201643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:20.210654 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cccfd84-4bed-4436-a27d-2b5401512d24 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:20.229640 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:20.250501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98fd1c-02fc-4cd9-aa63-d79914310b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:20.251690 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98fd1c-02fc-4cd9-aa63-d79914310b5b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:20.252117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98fd1c-02fc-4cd9-aa63-d79914310b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:20.252534 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98fd1c-02fc-4cd9-aa63-d79914310b5b None admin] PATH_INFO: `/v3/users/0e68e1214136444590de9239e1a0fc84` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:20.253872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98fd1c-02fc-4cd9-aa63-d79914310b5b None admin] RBAC: Authorizing `identity:get_user(user_id=0e68e1214136444590de9239e1a0fc84)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:20.255547 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98fd1c-02fc-4cd9-aa63-d79914310b5b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:20.257649 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 130/254] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:20 2022] GET /identity/v3/users/0e68e1214136444590de9239e1a0fc84 => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:20.263995 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98fd1c-02fc-4cd9-aa63-d79914310b5b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:20.277018 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1621f9-1de3-4932-a3b5-1d0df8b08b5b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:20.314741 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662527c-ba3d-4cd2-abfc-3eb9be5e723d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:20.315566 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662527c-ba3d-4cd2-abfc-3eb9be5e723d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:20.315857 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662527c-ba3d-4cd2-abfc-3eb9be5e723d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:20.316142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662527c-ba3d-4cd2-abfc-3eb9be5e723d None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:20.317016 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662527c-ba3d-4cd2-abfc-3eb9be5e723d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:20.317385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662527c-ba3d-4cd2-abfc-3eb9be5e723d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:20.318836 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662527c-ba3d-4cd2-abfc-3eb9be5e723d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:20.335438 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 125/255] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:20.340289 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662527c-ba3d-4cd2-abfc-3eb9be5e723d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:20.371647 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98fd1c-02fc-4cd9-aa63-d79914310b5b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:20.410496 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb23b9-a232-460d-ab69-da410195c6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:20.411675 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb23b9-a232-460d-ab69-da410195c6ff None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:20.412101 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb23b9-a232-460d-ab69-da410195c6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:20.412514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb23b9-a232-460d-ab69-da410195c6ff None admin] PATH_INFO: `/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:20.413857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb23b9-a232-460d-ab69-da410195c6ff None admin] RBAC: Authorizing `identity:get_project(project_id=c05a3987827c46a69dd1ab7b7f4b913b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:20.416066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb23b9-a232-460d-ab69-da410195c6ff None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:20.418000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 131/256] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:20 2022] GET /identity/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:20.425124 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb23b9-a232-460d-ab69-da410195c6ff None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:20.461141 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:20.466899 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662527c-ba3d-4cd2-abfc-3eb9be5e723d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:20.487154 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:20.511358 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c8385-2369-4d02-ba9e-6fef2a721d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:20.513598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c8385-2369-4d02-ba9e-6fef2a721d45 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:20.514530 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c8385-2369-4d02-ba9e-6fef2a721d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:20.515331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c8385-2369-4d02-ba9e-6fef2a721d45 None admin] PATH_INFO: `/v3/users/ac6a7b74505c418a884fc268487b7b8d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:20.517850 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c8385-2369-4d02-ba9e-6fef2a721d45 None admin] RBAC: Authorizing `identity:get_user(user_id=ac6a7b74505c418a884fc268487b7b8d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:20.520342 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c8385-2369-4d02-ba9e-6fef2a721d45 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:20.522513 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 126/257] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:20 2022] GET /identity/v3/users/ac6a7b74505c418a884fc268487b7b8d => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:20.528833 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0489d1a-a468-4bcc-b700-3c14f14d0f6a None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:20.529371 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0489d1a-a468-4bcc-b700-3c14f14d0f6a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:20.529938 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0489d1a-a468-4bcc-b700-3c14f14d0f6a None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:20.531695 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 127/258] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:20.533206 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0489d1a-a468-4bcc-b700-3c14f14d0f6a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:20.540490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb23b9-a232-460d-ab69-da410195c6ff None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:20.591246 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabdd597-29b6-4250-9bd8-71f494809c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:20.591246 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdd597-29b6-4250-9bd8-71f494809c0c None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:20.591246 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdd597-29b6-4250-9bd8-71f494809c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:20.591246 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdd597-29b6-4250-9bd8-71f494809c0c None admin] PATH_INFO: `/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b/users/0e68e1214136444590de9239e1a0fc84/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:20.591666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabdd597-29b6-4250-9bd8-71f494809c0c None admin] RBAC: Authorizing `identity:create_grant(project_id=c05a3987827c46a69dd1ab7b7f4b913b, user_id=0e68e1214136444590de9239e1a0fc84, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:20.592618 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabdd597-29b6-4250-9bd8-71f494809c0c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:20.614316 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 132/259] 10.222.0.81 () {58 vars in 1439 bytes} [Tue Jun 7 14:07:20 2022] PUT /identity/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b/users/0e68e1214136444590de9239e1a0fc84/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:07:20.614316 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3e14a-2c5d-4286-9002-0f8f43c47035 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:20.614316 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3e14a-2c5d-4286-9002-0f8f43c47035 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:20.614316 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3e14a-2c5d-4286-9002-0f8f43c47035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:20.636967 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-32c3e14a-2c5d-4286-9002-0f8f43c47035 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:20.638165 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-32c3e14a-2c5d-4286-9002-0f8f43c47035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:20.764066 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0489d1a-a468-4bcc-b700-3c14f14d0f6a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:20.788330 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 133/260] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:20.830220 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3e14a-2c5d-4286-9002-0f8f43c47035 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:20.857789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6599773-21d7-409c-987f-9dd4b8724035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:20.858476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6599773-21d7-409c-987f-9dd4b8724035 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:20.858763 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6599773-21d7-409c-987f-9dd4b8724035 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:20.859040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6599773-21d7-409c-987f-9dd4b8724035 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:20.923689 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:20.965637 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:20.978785 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6599773-21d7-409c-987f-9dd4b8724035 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:20.981748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6599773-21d7-409c-987f-9dd4b8724035 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:21.120367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 128/261] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 14:07:21.126049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6599773-21d7-409c-987f-9dd4b8724035 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:21.169239 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3e14a-2c5d-4286-9002-0f8f43c47035 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:21.210769 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:21.264964 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6599773-21d7-409c-987f-9dd4b8724035 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:21.270515 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8b8fe-6269-4c3f-85eb-6711ddb876aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:21.276986 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8b8fe-6269-4c3f-85eb-6711ddb876aa None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:21.277513 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8b8fe-6269-4c3f-85eb-6711ddb876aa None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:21.277991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8b8fe-6269-4c3f-85eb-6711ddb876aa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:21.298444 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a8b8fe-6269-4c3f-85eb-6711ddb876aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:21.314961 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-94a8b8fe-6269-4c3f-85eb-6711ddb876aa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:07:21.314961 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:21.314961 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:21.314961 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:21.314961 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:21.314961 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:21.314961 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:21.314961 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:21.314961 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:21.314961 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:21.314961 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:21.314961 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:21.314961 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:21.314961 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:21.314961 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:21.314961 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:21.315942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:21.315942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:21.315942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:21.315942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:21.315942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:21.315942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:21.315942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:21.315942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:21.315942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:21.315942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:21.315942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:21.315942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:21.315942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:21.315942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:21.315942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:21.315942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:21.316742 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:21.316742 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:21.316742 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:21.316742 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:21.316742 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:21.316742 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:21.316742 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:21.316742 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:21.316742 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:21.316742 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:21.316742 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:07:21.316742 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:21.325308 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 134/262] 10.222.0.81 () {56 vars in 1114 bytes} [Tue Jun 7 14:07:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 495 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:07:21.329435 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8b8fe-6269-4c3f-85eb-6711ddb876aa None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:21.346873 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a1c4f-8aaa-46e3-bd18-74af44cda994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:21.348132 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a1c4f-8aaa-46e3-bd18-74af44cda994 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:21.348621 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a1c4f-8aaa-46e3-bd18-74af44cda994 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:21.350464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a1c4f-8aaa-46e3-bd18-74af44cda994 None admin] PATH_INFO: `/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:21.352919 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a1c4f-8aaa-46e3-bd18-74af44cda994 None admin] RBAC: Authorizing `identity:get_project(project_id=c05a3987827c46a69dd1ab7b7f4b913b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:21.354600 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a1c4f-8aaa-46e3-bd18-74af44cda994 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:21.358341 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 129/263] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:21 2022] GET /identity/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:21.364498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a1c4f-8aaa-46e3-bd18-74af44cda994 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:21.530381 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8b8fe-6269-4c3f-85eb-6711ddb876aa None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:21.533239 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:21.578229 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a1c4f-8aaa-46e3-bd18-74af44cda994 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:21.580944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42bb6f5-7bf0-4670-af7c-fd91fa88b1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:21.581636 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42bb6f5-7bf0-4670-af7c-fd91fa88b1ef None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:21.581921 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42bb6f5-7bf0-4670-af7c-fd91fa88b1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:21.582194 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42bb6f5-7bf0-4670-af7c-fd91fa88b1ef None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:21.582952 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42bb6f5-7bf0-4670-af7c-fd91fa88b1ef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:21.583275 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42bb6f5-7bf0-4670-af7c-fd91fa88b1ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:21.584672 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42bb6f5-7bf0-4670-af7c-fd91fa88b1ef None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:21.602824 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 135/264] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:07:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:21.610758 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42bb6f5-7bf0-4670-af7c-fd91fa88b1ef None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:21.613167 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11ecdc-f838-4f53-a5c9-f2a95beaee56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:21.613841 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11ecdc-f838-4f53-a5c9-f2a95beaee56 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:21.614127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11ecdc-f838-4f53-a5c9-f2a95beaee56 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:21.614402 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11ecdc-f838-4f53-a5c9-f2a95beaee56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:21.615258 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f11ecdc-f838-4f53-a5c9-f2a95beaee56 None admin] RBAC: Adding query filter params (role.id=82bbe3651b9b4f378fd150d93357b2b1, scope.project.id=c05a3987827c46a69dd1ab7b7f4b913b, user.id=ac6a7b74505c418a884fc268487b7b8d) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:21.615578 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f11ecdc-f838-4f53-a5c9-f2a95beaee56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:21.617043 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f11ecdc-f838-4f53-a5c9-f2a95beaee56 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:21.624633 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 130/265] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:07:21 2022] GET /identity/v3/role_assignments?user.id=ac6a7b74505c418a884fc268487b7b8d&scope.project.id=c05a3987827c46a69dd1ab7b7f4b913b&role.id=82bbe3651b9b4f378fd150d93357b2b1 => generated 608 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:21.625195 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:21.766953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42bb6f5-7bf0-4670-af7c-fd91fa88b1ef None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:21.811119 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa71328-6cf9-4994-a644-e247e2b77379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:21.812422 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa71328-6cf9-4994-a644-e247e2b77379 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:21.812939 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa71328-6cf9-4994-a644-e247e2b77379 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:21.813436 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa71328-6cf9-4994-a644-e247e2b77379 None admin] PATH_INFO: `/v3/users/ac6a7b74505c418a884fc268487b7b8d` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:21.814921 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa71328-6cf9-4994-a644-e247e2b77379 None admin] RBAC: Authorizing `identity:get_user(user_id=ac6a7b74505c418a884fc268487b7b8d)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:21.816621 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa71328-6cf9-4994-a644-e247e2b77379 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:21.819158 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 136/266] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:21 2022] GET /identity/v3/users/ac6a7b74505c418a884fc268487b7b8d => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:21.827259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11ecdc-f838-4f53-a5c9-f2a95beaee56 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:21.934843 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11ecdc-f838-4f53-a5c9-f2a95beaee56 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:21.968723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365ccb2-6367-4191-aaff-debc54e2d921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:21.969380 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365ccb2-6367-4191-aaff-debc54e2d921 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:21.969664 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365ccb2-6367-4191-aaff-debc54e2d921 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:21.969941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365ccb2-6367-4191-aaff-debc54e2d921 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:22.035809 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:22.041162 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3365ccb2-6367-4191-aaff-debc54e2d921 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:22.042848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3365ccb2-6367-4191-aaff-debc54e2d921 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:22.147550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 131/267] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:07:22.155000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa71328-6cf9-4994-a644-e247e2b77379 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:22.198817 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:22.282955 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa71328-6cf9-4994-a644-e247e2b77379 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:22.323651 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a6490-3b82-4036-8659-4b4dbcadaa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:22.324948 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a6490-3b82-4036-8659-4b4dbcadaa03 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:22.325395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a6490-3b82-4036-8659-4b4dbcadaa03 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:22.325757 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a6490-3b82-4036-8659-4b4dbcadaa03 None admin] PATH_INFO: `/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:22.326997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a6490-3b82-4036-8659-4b4dbcadaa03 None admin] RBAC: Authorizing `identity:get_project(project_id=c05a3987827c46a69dd1ab7b7f4b913b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:22.328468 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a6490-3b82-4036-8659-4b4dbcadaa03 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:22.330502 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 137/268] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:22 2022] GET /identity/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:22.335929 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365ccb2-6367-4191-aaff-debc54e2d921 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:22.459996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:22.462768 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365ccb2-6367-4191-aaff-debc54e2d921 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:22.498945 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0994fe1-4d11-440a-bd75-6e11c84bf2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:22.499639 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0994fe1-4d11-440a-bd75-6e11c84bf2cb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:22.499923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0994fe1-4d11-440a-bd75-6e11c84bf2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:22.500197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0994fe1-4d11-440a-bd75-6e11c84bf2cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:22.501033 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0994fe1-4d11-440a-bd75-6e11c84bf2cb None admin] RBAC: Adding query filter params (role.id=6d02db1fe03b447c9e519f4f2aa16574, scope.project.id=c05a3987827c46a69dd1ab7b7f4b913b, user.id=ac6a7b74505c418a884fc268487b7b8d) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:22.501345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0994fe1-4d11-440a-bd75-6e11c84bf2cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:22.502799 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0994fe1-4d11-440a-bd75-6e11c84bf2cb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:22.509496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 132/269] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:07:22 2022] GET /identity/v3/role_assignments?user.id=ac6a7b74505c418a884fc268487b7b8d&scope.project.id=c05a3987827c46a69dd1ab7b7f4b913b&role.id=6d02db1fe03b447c9e519f4f2aa16574 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:22.984062 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199c849-fac0-4d0b-9e2f-a4f8ba72608c None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:22.984596 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199c849-fac0-4d0b-9e2f-a4f8ba72608c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:22.985022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199c849-fac0-4d0b-9e2f-a4f8ba72608c None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:22.986614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 138/270] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:22.993429 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2b6c2-1904-4dfd-b470-8302681d5fb9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:22.993993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2b6c2-1904-4dfd-b470-8302681d5fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:22.994168 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2b6c2-1904-4dfd-b470-8302681d5fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:22.997776 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec93ed7-abef-4c03-a15e-f357a5549ad5 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:22.998380 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec93ed7-abef-4c03-a15e-f357a5549ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:22.998882 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec93ed7-abef-4c03-a15e-f357a5549ad5 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:23.001037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 139/271] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:23.007532 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f43c2-3738-423b-82b1-053be5fb6317 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:23.008150 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f43c2-3738-423b-82b1-053be5fb6317 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:23.009249 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f43c2-3738-423b-82b1-053be5fb6317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:23.019789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-96b2b6c2-1904-4dfd-b470-8302681d5fb9 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:23.020993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-96b2b6c2-1904-4dfd-b470-8302681d5fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:23.029062 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-e26f43c2-3738-423b-82b1-053be5fb6317 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:23.030928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e26f43c2-3738-423b-82b1-053be5fb6317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:23.108447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 133/272] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:23.119322 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 140/273] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:23.132798 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2b6c2-1904-4dfd-b470-8302681d5fb9 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:23.139104 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26f43c2-3738-423b-82b1-053be5fb6317 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:23.172330 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:23.237276 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:23.247427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2b6c2-1904-4dfd-b470-8302681d5fb9 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:23.251807 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26f43c2-3738-423b-82b1-053be5fb6317 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:23.285745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5f34b-cde0-4629-ab03-236adfccbfa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:23.286841 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5f34b-cde0-4629-ab03-236adfccbfa9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:23.287319 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5f34b-cde0-4629-ab03-236adfccbfa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:23.287785 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5f34b-cde0-4629-ab03-236adfccbfa9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:23.290680 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7a95f-7bc5-4158-9ac0-955b47d96045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:23.292157 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7a95f-7bc5-4158-9ac0-955b47d96045 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:23.292590 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7a95f-7bc5-4158-9ac0-955b47d96045 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:23.293011 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7a95f-7bc5-4158-9ac0-955b47d96045 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:23.299913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d5f34b-cde0-4629-ab03-236adfccbfa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:23.304555 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f7a95f-7bc5-4158-9ac0-955b47d96045 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:23.308393 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-45d5f34b-cde0-4629-ab03-236adfccbfa9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:07:23.308393 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:23.308393 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:23.308393 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:23.308393 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:23.308393 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:23.308393 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:23.308393 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:23.308393 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:23.308393 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:23.308393 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:23.308393 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:23.308393 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:23.308393 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:23.308393 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:23.308393 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:23.309279 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:23.309279 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:23.309279 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:23.309279 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:23.309279 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:23.309279 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:23.309279 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:23.309279 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:23.309279 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:23.309279 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:23.309279 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:23.309279 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:23.309279 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:23.309279 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:23.309279 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:23.309279 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:23.310270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:23.310270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:23.310270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:23.310270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:23.310270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:23.310270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:23.310270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:23.310270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:23.310270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:23.310270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:23.310270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:07:23.310270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:23.310270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 134/274] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:23.315281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5f34b-cde0-4629-ab03-236adfccbfa9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:23.316789 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-53f7a95f-7bc5-4158-9ac0-955b47d96045 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:23.316789 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:23.316789 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:23.316789 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:23.316789 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:23.316789 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:23.316789 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:23.316789 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:23.316789 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:23.316789 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:23.316789 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:23.316789 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:23.316789 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:23.316789 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:23.316789 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:23.316789 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:23.317872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:23.317872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:23.317872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:23.317872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:23.317872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:23.317872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:23.317872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:23.317872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:23.317872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:23.317872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:23.317872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:23.317872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:23.317872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:23.317872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:23.317872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:23.317872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:23.318873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:23.318873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:23.318873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:23.318873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:23.318873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:23.318873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:23.318873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:23.318873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:23.318873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:23.318873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:23.318873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:23.318873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:23.326409 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 141/275] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:07:23.327977 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7a95f-7bc5-4158-9ac0-955b47d96045 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:23.425483 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5f34b-cde0-4629-ab03-236adfccbfa9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:23.432055 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:23.442296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7a95f-7bc5-4158-9ac0-955b47d96045 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:23.459876 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b305d-79e5-4699-a973-54aaa1851ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:23.460541 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b305d-79e5-4699-a973-54aaa1851ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:23.460821 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b305d-79e5-4699-a973-54aaa1851ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:23.461092 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b305d-79e5-4699-a973-54aaa1851ac9 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:23.462671 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b305d-79e5-4699-a973-54aaa1851ac9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:23.463122 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b305d-79e5-4699-a973-54aaa1851ac9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:23.464592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b305d-79e5-4699-a973-54aaa1851ac9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:23.473032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 135/276] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:07:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:23.474169 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb624872-d16b-4198-b666-6c5e18fb2490 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:23.474452 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb624872-d16b-4198-b666-6c5e18fb2490 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:23.474723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb624872-d16b-4198-b666-6c5e18fb2490 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:23.476158 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 136/277] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:23.481200 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb624872-d16b-4198-b666-6c5e18fb2490 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:23.482864 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084bf403-b735-4314-ab7c-ac24c7bb4ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:23.483590 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bf403-b735-4314-ab7c-ac24c7bb4ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:23.483977 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bf403-b735-4314-ab7c-ac24c7bb4ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:23.484256 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bf403-b735-4314-ab7c-ac24c7bb4ce0 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:23.485127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084bf403-b735-4314-ab7c-ac24c7bb4ce0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:23.485432 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084bf403-b735-4314-ab7c-ac24c7bb4ce0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:23.486856 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084bf403-b735-4314-ab7c-ac24c7bb4ce0 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:23.502141 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 142/278] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:07:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:23.502141 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad261009-06c3-4c3c-bd11-f5a5516aa4f1 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:23.502141 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad261009-06c3-4c3c-bd11-f5a5516aa4f1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:23.502141 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad261009-06c3-4c3c-bd11-f5a5516aa4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:23.504564 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:23.520025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-ad261009-06c3-4c3c-bd11-f5a5516aa4f1 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:23.521252 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ad261009-06c3-4c3c-bd11-f5a5516aa4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:23.592605 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 143/279] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:23.600916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad261009-06c3-4c3c-bd11-f5a5516aa4f1 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:23.603979 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb624872-d16b-4198-b666-6c5e18fb2490 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:23.640502 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbf38d-52cd-453b-9d03-f3591a3ec14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:23.641486 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbf38d-52cd-453b-9d03-f3591a3ec14d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:23.641775 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbf38d-52cd-453b-9d03-f3591a3ec14d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:23.642047 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbf38d-52cd-453b-9d03-f3591a3ec14d None admin] PATH_INFO: `/v3/users/da9e8f9fd6c54493bdca6f956c2eb20f` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:23.643249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fbf38d-52cd-453b-9d03-f3591a3ec14d None admin] RBAC: Authorizing `identity:get_user(user_id=da9e8f9fd6c54493bdca6f956c2eb20f)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:23.644693 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fbf38d-52cd-453b-9d03-f3591a3ec14d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:23.646336 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 137/280] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:23 2022] GET /identity/v3/users/da9e8f9fd6c54493bdca6f956c2eb20f => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:23.646336 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbf38d-52cd-453b-9d03-f3591a3ec14d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:23.701959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:23.720433 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad261009-06c3-4c3c-bd11-f5a5516aa4f1 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:23.735101 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:23.757057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbf38d-52cd-453b-9d03-f3591a3ec14d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:23.760226 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536434cf-926d-4b5a-8379-9f8498b58ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:23.761110 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536434cf-926d-4b5a-8379-9f8498b58ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:23.761393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536434cf-926d-4b5a-8379-9f8498b58ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:23.761669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536434cf-926d-4b5a-8379-9f8498b58ce0 None admin] PATH_INFO: `/v3/users/cad6af88a0844139adb20477a10e13c4` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:23.777171 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536434cf-926d-4b5a-8379-9f8498b58ce0 None admin] RBAC: Authorizing `identity:get_user(user_id=cad6af88a0844139adb20477a10e13c4)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:23.778618 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536434cf-926d-4b5a-8379-9f8498b58ce0 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:23.780360 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 144/281] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:23 2022] GET /identity/v3/users/cad6af88a0844139adb20477a10e13c4 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:23.781122 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536434cf-926d-4b5a-8379-9f8498b58ce0 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:23.797748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a5a09-5586-46ce-9314-a4cb242a4e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:23.798805 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a5a09-5586-46ce-9314-a4cb242a4e73 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:23.799290 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a5a09-5586-46ce-9314-a4cb242a4e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:23.799743 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a5a09-5586-46ce-9314-a4cb242a4e73 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:23.801100 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a5a09-5586-46ce-9314-a4cb242a4e73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:23.802778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a5a09-5586-46ce-9314-a4cb242a4e73 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:23.804609 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 138/282] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:07:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:23.810530 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a5a09-5586-46ce-9314-a4cb242a4e73 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:23.884926 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536434cf-926d-4b5a-8379-9f8498b58ce0 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:23.911910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a5a09-5586-46ce-9314-a4cb242a4e73 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:23.919006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb59fa3-a4c7-4686-ae92-07f462873890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:23.919898 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb59fa3-a4c7-4686-ae92-07f462873890 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:23.920180 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb59fa3-a4c7-4686-ae92-07f462873890 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:23.920452 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb59fa3-a4c7-4686-ae92-07f462873890 None admin] PATH_INFO: `/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:23.921644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb59fa3-a4c7-4686-ae92-07f462873890 None admin] RBAC: Authorizing `identity:get_project(project_id=c05a3987827c46a69dd1ab7b7f4b913b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:23.923085 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb59fa3-a4c7-4686-ae92-07f462873890 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:23.924848 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 145/283] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:23 2022] GET /identity/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:23.925592 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb59fa3-a4c7-4686-ae92-07f462873890 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:23.937435 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:23.951705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bd632-079a-4dd8-9bdb-20315600a960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:23.952362 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bd632-079a-4dd8-9bdb-20315600a960 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:23.952645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bd632-079a-4dd8-9bdb-20315600a960 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:23.952920 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bd632-079a-4dd8-9bdb-20315600a960 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:23.953748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8bd632-079a-4dd8-9bdb-20315600a960 None admin] RBAC: Adding query filter params (role.id=ca91f64e7a46431c8aa4285c4620f216, user.id=cad6af88a0844139adb20477a10e13c4, scope.system=all) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:23.954060 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8bd632-079a-4dd8-9bdb-20315600a960 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:23.955520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8bd632-079a-4dd8-9bdb-20315600a960 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:23.960377 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 139/284] 10.222.0.81 () {56 vars in 1308 bytes} [Tue Jun 7 14:07:23 2022] GET /identity/v3/role_assignments?user.id=cad6af88a0844139adb20477a10e13c4&scope.system=all&role.id=ca91f64e7a46431c8aa4285c4620f216 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:23.961671 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70693ecc-d7f4-4520-9a16-1992420d6e9e None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:23.961958 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70693ecc-d7f4-4520-9a16-1992420d6e9e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:23.962225 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70693ecc-d7f4-4520-9a16-1992420d6e9e None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:23.967395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 140/285] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:23.967395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70693ecc-d7f4-4520-9a16-1992420d6e9e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:23.989901 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:24.036794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb59fa3-a4c7-4686-ae92-07f462873890 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:24.079420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69cf3ef-b865-4cd3-9a8d-04c90611f796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:24.080456 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cf3ef-b865-4cd3-9a8d-04c90611f796 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:24.080741 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cf3ef-b865-4cd3-9a8d-04c90611f796 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:24.081015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cf3ef-b865-4cd3-9a8d-04c90611f796 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:24.081739 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69cf3ef-b865-4cd3-9a8d-04c90611f796 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:24.083246 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69cf3ef-b865-4cd3-9a8d-04c90611f796 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:24.098934 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70693ecc-d7f4-4520-9a16-1992420d6e9e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:24.108115 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 146/286] 10.222.0.81 () {60 vars in 1132 bytes} [Tue Jun 7 14:07:23 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:07:24.109893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21eb00-519a-448e-89a5-c0d674b9f19f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:24.110346 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21eb00-519a-448e-89a5-c0d674b9f19f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:24.110757 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21eb00-519a-448e-89a5-c0d674b9f19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:24.130735 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-1b21eb00-519a-448e-89a5-c0d674b9f19f None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:24.132282 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1b21eb00-519a-448e-89a5-c0d674b9f19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:24.159382 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38fd873-b8c4-4769-bbe0-29faba64c08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:24.160980 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fd873-b8c4-4769-bbe0-29faba64c08e None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:24.161264 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fd873-b8c4-4769-bbe0-29faba64c08e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:24.161532 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fd873-b8c4-4769-bbe0-29faba64c08e None admin] PATH_INFO: `/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b/users/da9e8f9fd6c54493bdca6f956c2eb20f/roles/a1d21c4d26ef419298e001001f4655b7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:24.167540 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38fd873-b8c4-4769-bbe0-29faba64c08e None admin] RBAC: Authorizing `identity:create_grant(project_id=c05a3987827c46a69dd1ab7b7f4b913b, user_id=da9e8f9fd6c54493bdca6f956c2eb20f, role_id=a1d21c4d26ef419298e001001f4655b7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:24.167540 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38fd873-b8c4-4769-bbe0-29faba64c08e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:24.187268 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 141/287] 10.222.0.81 () {58 vars in 1439 bytes} [Tue Jun 7 14:07:23 2022] PUT /identity/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b/users/da9e8f9fd6c54493bdca6f956c2eb20f/roles/a1d21c4d26ef419298e001001f4655b7 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:07:24.223388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:24.234243 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 147/288] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:24.259071 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38fd873-b8c4-4769-bbe0-29faba64c08e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:24.357192 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:24.389281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38fd873-b8c4-4769-bbe0-29faba64c08e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:24.426901 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f8c9d1-f03d-45dd-ba61-b8764c557c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:24.427741 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8c9d1-f03d-45dd-ba61-b8764c557c8c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:24.428027 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8c9d1-f03d-45dd-ba61-b8764c557c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:24.428304 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8c9d1-f03d-45dd-ba61-b8764c557c8c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:24.440688 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f8c9d1-f03d-45dd-ba61-b8764c557c8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:24.449563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-54f8c9d1-f03d-45dd-ba61-b8764c557c8c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:24.449563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:24.449563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:24.449563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:24.449563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:24.449563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:24.449563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:24.449563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:24.449563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:24.449563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:24.449563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:24.449563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:24.449563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:24.449563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:24.449563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:24.449563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:24.450464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:24.450464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:24.450464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:24.450464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:24.450464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:24.450464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:24.450464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:24.450464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:24.450464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:24.450464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:24.450464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:24.450464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:24.450464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:24.450464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:24.450464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:24.450464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:24.451702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:24.451702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:24.451702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:24.451702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:24.451702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:24.451702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:24.451702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:24.451702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:24.451702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:24.451702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:24.451702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:24.451702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:24.451702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 142/289] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:24.456938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21eb00-519a-448e-89a5-c0d674b9f19f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:24.584855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21eb00-519a-448e-89a5-c0d674b9f19f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:24.631983 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc62c6-13f2-49d6-8f67-3e6ef22a5c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:24.632580 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc62c6-13f2-49d6-8f67-3e6ef22a5c0a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:24.632860 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc62c6-13f2-49d6-8f67-3e6ef22a5c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:24.633134 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc62c6-13f2-49d6-8f67-3e6ef22a5c0a None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:24.634627 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc62c6-13f2-49d6-8f67-3e6ef22a5c0a None admin] RBAC: Adding query filter params (name=member) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:24.635034 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc62c6-13f2-49d6-8f67-3e6ef22a5c0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:24.636457 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc62c6-13f2-49d6-8f67-3e6ef22a5c0a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:24.645970 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 148/290] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:07:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:24.651662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f8c9d1-f03d-45dd-ba61-b8764c557c8c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:24.763979 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f8c9d1-f03d-45dd-ba61-b8764c557c8c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:24.811968 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947db61b-d17b-46c9-8b28-1709434763b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:24.812851 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947db61b-d17b-46c9-8b28-1709434763b2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:24.813134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947db61b-d17b-46c9-8b28-1709434763b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:24.813411 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947db61b-d17b-46c9-8b28-1709434763b2 None admin] PATH_INFO: `/v3/users/0e68e1214136444590de9239e1a0fc84` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:24.819948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947db61b-d17b-46c9-8b28-1709434763b2 None admin] RBAC: Authorizing `identity:get_user(user_id=0e68e1214136444590de9239e1a0fc84)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:24.821994 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947db61b-d17b-46c9-8b28-1709434763b2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:24.824003 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 143/291] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:24 2022] GET /identity/v3/users/0e68e1214136444590de9239e1a0fc84 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:24.829327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc62c6-13f2-49d6-8f67-3e6ef22a5c0a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:24.937634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:24.994809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc62c6-13f2-49d6-8f67-3e6ef22a5c0a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:25.049327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee860f4-c21f-4e29-8fc2-51dc06f7f10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:25.050262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee860f4-c21f-4e29-8fc2-51dc06f7f10a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:25.050684 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee860f4-c21f-4e29-8fc2-51dc06f7f10a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:25.051095 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee860f4-c21f-4e29-8fc2-51dc06f7f10a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:25.137411 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee860f4-c21f-4e29-8fc2-51dc06f7f10a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:25.138874 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee860f4-c21f-4e29-8fc2-51dc06f7f10a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:25.198754 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:25.245779 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 149/292] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:07:25.252023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947db61b-d17b-46c9-8b28-1709434763b2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:25.254937 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:25.392260 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947db61b-d17b-46c9-8b28-1709434763b2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:25.437923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c7569-f8a3-48f8-8d59-2aaa0eb3dd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:25.439150 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c7569-f8a3-48f8-8d59-2aaa0eb3dd1f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:25.439651 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c7569-f8a3-48f8-8d59-2aaa0eb3dd1f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:25.440110 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c7569-f8a3-48f8-8d59-2aaa0eb3dd1f None admin] PATH_INFO: `/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:25.443929 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c7569-f8a3-48f8-8d59-2aaa0eb3dd1f None admin] RBAC: Authorizing `identity:get_project(project_id=c05a3987827c46a69dd1ab7b7f4b913b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:25.445613 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c7569-f8a3-48f8-8d59-2aaa0eb3dd1f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:25.448557 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 144/293] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:25 2022] GET /identity/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:25.454502 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee860f4-c21f-4e29-8fc2-51dc06f7f10a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:25.581838 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee860f4-c21f-4e29-8fc2-51dc06f7f10a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:25.622834 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e514f78-4feb-4653-b2ca-711112704da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:25.623521 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e514f78-4feb-4653-b2ca-711112704da6 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:25.623802 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e514f78-4feb-4653-b2ca-711112704da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:25.624075 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e514f78-4feb-4653-b2ca-711112704da6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:25.624897 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e514f78-4feb-4653-b2ca-711112704da6 None admin] RBAC: Adding query filter params (role.id=ca91f64e7a46431c8aa4285c4620f216, scope.project.id=c05a3987827c46a69dd1ab7b7f4b913b, user.id=0e68e1214136444590de9239e1a0fc84) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:25.625207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e514f78-4feb-4653-b2ca-711112704da6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:25.626619 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e514f78-4feb-4653-b2ca-711112704da6 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:25.632723 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 150/294] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:07:25 2022] GET /identity/v3/role_assignments?user.id=0e68e1214136444590de9239e1a0fc84&scope.project.id=c05a3987827c46a69dd1ab7b7f4b913b&role.id=ca91f64e7a46431c8aa4285c4620f216 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:26.567542 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffc3a8-08ef-4829-b594-60f01d255978 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:26.568048 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffc3a8-08ef-4829-b594-60f01d255978 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:26.568160 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffc3a8-08ef-4829-b594-60f01d255978 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:26.569698 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 145/295] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:26.575055 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220df3e-02f4-45d1-a114-038f4a3e77b6 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:26.575624 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220df3e-02f4-45d1-a114-038f4a3e77b6 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:26.576134 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220df3e-02f4-45d1-a114-038f4a3e77b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:26.593510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-8220df3e-02f4-45d1-a114-038f4a3e77b6 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:26.595494 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8220df3e-02f4-45d1-a114-038f4a3e77b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:26.614102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:26.672150 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 151/296] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:26.695237 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbffc3a8-08ef-4829-b594-60f01d255978 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:26.770695 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:26.803833 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbffc3a8-08ef-4829-b594-60f01d255978 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:26.810565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf05d16-b1f8-465b-aad2-3e8302c8e05b None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:26.810843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf05d16-b1f8-465b-aad2-3e8302c8e05b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:26.811120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf05d16-b1f8-465b-aad2-3e8302c8e05b None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:26.813014 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 152/297] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:07:26.818654 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc216fda-76a2-41bb-b6d3-4b4663d04301 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:26.819198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc216fda-76a2-41bb-b6d3-4b4663d04301 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:26.819756 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc216fda-76a2-41bb-b6d3-4b4663d04301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:26.839162 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-cc216fda-76a2-41bb-b6d3-4b4663d04301 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:26.840724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-cc216fda-76a2-41bb-b6d3-4b4663d04301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:26.844256 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8900635-2da1-45cc-9475-7acf6489fffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:26.845074 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8900635-2da1-45cc-9475-7acf6489fffc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:26.845362 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8900635-2da1-45cc-9475-7acf6489fffc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:26.845643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8900635-2da1-45cc-9475-7acf6489fffc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:26.857406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8900635-2da1-45cc-9475-7acf6489fffc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:26.865349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-e8900635-2da1-45cc-9475-7acf6489fffc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:26.865349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:26.865349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:26.865349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:26.865349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:26.865349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:26.865349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:26.865349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:26.865349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:26.865349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:26.865349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:26.865349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:26.865349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:26.865349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:26.865349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:26.865349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:26.866227 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:26.866227 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:26.866227 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:26.866227 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:26.866227 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:26.866227 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:26.866227 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:26.866227 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:26.866227 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:26.866227 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:26.866227 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:26.866227 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:26.866227 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:26.866227 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:26.866227 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:26.866227 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:26.867041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:26.867041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:26.867041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:26.867041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:26.867041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:26.867041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:26.867041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:26.867041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:26.867041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:26.867041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:26.867041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:26.867041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:26.867041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 146/298] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:26.874494 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8900635-2da1-45cc-9475-7acf6489fffc None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:26.930088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 153/299] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:26.959284 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc216fda-76a2-41bb-b6d3-4b4663d04301 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:27.005877 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8900635-2da1-45cc-9475-7acf6489fffc None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:27.036895 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:27.044015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa572dc-e995-4992-8cdb-51049d52a591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:27.044689 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa572dc-e995-4992-8cdb-51049d52a591 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:27.044967 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa572dc-e995-4992-8cdb-51049d52a591 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:27.045242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa572dc-e995-4992-8cdb-51049d52a591 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:27.046059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa572dc-e995-4992-8cdb-51049d52a591 None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:27.046300 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa572dc-e995-4992-8cdb-51049d52a591 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:27.047696 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa572dc-e995-4992-8cdb-51049d52a591 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:27.048836 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:27.060902 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 147/300] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:07:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:27.060902 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1744cf-1b34-4754-8d02-540ef73e5112 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:27.060902 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1744cf-1b34-4754-8d02-540ef73e5112 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:27.060902 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1744cf-1b34-4754-8d02-540ef73e5112 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:27.060902 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 148/301] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:27.061948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1744cf-1b34-4754-8d02-540ef73e5112 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:27.076865 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc216fda-76a2-41bb-b6d3-4b4663d04301 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:27.113938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9628a9a-9085-4c0c-a3c6-681bfb3915f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:27.114896 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9628a9a-9085-4c0c-a3c6-681bfb3915f3 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:27.115179 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9628a9a-9085-4c0c-a3c6-681bfb3915f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:27.115493 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9628a9a-9085-4c0c-a3c6-681bfb3915f3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:27.126417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9628a9a-9085-4c0c-a3c6-681bfb3915f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:27.134933 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-e9628a9a-9085-4c0c-a3c6-681bfb3915f3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:07:27.134933 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:27.134933 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:27.134933 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:27.134933 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:27.134933 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:27.134933 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:27.134933 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:27.134933 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:27.134933 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:27.134933 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:27.134933 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:27.134933 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:27.134933 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:27.134933 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:27.134933 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:27.136036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:27.136036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:27.136036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:27.136036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:27.136036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:27.136036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:27.136036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:27.136036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:27.136036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:27.136036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:27.136036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:27.136036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:27.136036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:27.136036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:27.136036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:27.136036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:27.136869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:27.136869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:27.136869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:27.136869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:27.136869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:27.136869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:27.136869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:27.136869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:27.136869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:27.136869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:27.136869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:07:27.136869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:27.136869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 154/302] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:07:27.137766 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aaa3dd-207d-4064-97b0-6a729de6192d None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:27.138049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aaa3dd-207d-4064-97b0-6a729de6192d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:27.138316 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aaa3dd-207d-4064-97b0-6a729de6192d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:27.156231 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a2aaa3dd-207d-4064-97b0-6a729de6192d None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:27.157402 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a2aaa3dd-207d-4064-97b0-6a729de6192d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:27.184153 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1744cf-1b34-4754-8d02-540ef73e5112 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:27.220404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407219d5-feb4-4272-a3ff-4873f1887b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:27.221254 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407219d5-feb4-4272-a3ff-4873f1887b69 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:27.221535 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407219d5-feb4-4272-a3ff-4873f1887b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:27.221805 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407219d5-feb4-4272-a3ff-4873f1887b69 None admin] PATH_INFO: `/v3/users/cad6af88a0844139adb20477a10e13c4` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:27.222986 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407219d5-feb4-4272-a3ff-4873f1887b69 None admin] RBAC: Authorizing `identity:get_user(user_id=cad6af88a0844139adb20477a10e13c4)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:27.224440 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407219d5-feb4-4272-a3ff-4873f1887b69 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:27.226047 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 149/303] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:27 2022] GET /identity/v3/users/cad6af88a0844139adb20477a10e13c4 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:27.227380 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407219d5-feb4-4272-a3ff-4873f1887b69 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:27.235249 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 155/304] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:27.235249 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aaa3dd-207d-4064-97b0-6a729de6192d None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:27.299487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:27.305855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:27.335788 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aaa3dd-207d-4064-97b0-6a729de6192d None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:27.341468 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407219d5-feb4-4272-a3ff-4873f1887b69 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:27.370091 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952c2df-7e22-482a-b124-9c1cbfc6f028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:27.370952 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952c2df-7e22-482a-b124-9c1cbfc6f028 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:27.371250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952c2df-7e22-482a-b124-9c1cbfc6f028 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:27.371531 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952c2df-7e22-482a-b124-9c1cbfc6f028 None admin] PATH_INFO: `/v3/system/users/cad6af88a0844139adb20477a10e13c4/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:27.373244 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f952c2df-7e22-482a-b124-9c1cbfc6f028 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cad6af88a0844139adb20477a10e13c4, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:27.374460 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a1e64-4eb9-4d66-98ed-01b746ec1024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:27.374637 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f952c2df-7e22-482a-b124-9c1cbfc6f028 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:27.375133 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a1e64-4eb9-4d66-98ed-01b746ec1024 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:27.375455 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a1e64-4eb9-4d66-98ed-01b746ec1024 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:27.375737 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a1e64-4eb9-4d66-98ed-01b746ec1024 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:27.376492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a1e64-4eb9-4d66-98ed-01b746ec1024 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:27.376801 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a1e64-4eb9-4d66-98ed-01b746ec1024 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:27.378265 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a1e64-4eb9-4d66-98ed-01b746ec1024 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:27.382676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 156/305] 10.222.0.81 () {58 vars in 1334 bytes} [Tue Jun 7 14:07:27 2022] PUT /identity/v3/system/users/cad6af88a0844139adb20477a10e13c4/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:07:27.384760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5ea3a-e69c-41a0-a1c9-73c0445c1ff6 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:27.385158 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5ea3a-e69c-41a0-a1c9-73c0445c1ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:27.385445 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5ea3a-e69c-41a0-a1c9-73c0445c1ff6 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:27.390968 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 157/306] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:27.391165 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 150/307] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:07:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:27.391296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5ea3a-e69c-41a0-a1c9-73c0445c1ff6 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:27.394257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0385a-95bf-4626-94dd-04c2085ff698 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:27.394583 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0385a-95bf-4626-94dd-04c2085ff698 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:27.394864 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0385a-95bf-4626-94dd-04c2085ff698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:27.413132 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c9b0385a-95bf-4626-94dd-04c2085ff698 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:27.414368 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c9b0385a-95bf-4626-94dd-04c2085ff698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:27.498880 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 151/308] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:27.500179 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0385a-95bf-4626-94dd-04c2085ff698 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:27.541522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5ea3a-e69c-41a0-a1c9-73c0445c1ff6 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:27.564525 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:27.582424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79a5ed-4ff0-49b6-b429-31894d0db9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:27.583244 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79a5ed-4ff0-49b6-b429-31894d0db9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:27.583546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79a5ed-4ff0-49b6-b429-31894d0db9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:27.583826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79a5ed-4ff0-49b6-b429-31894d0db9a1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:27.586825 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:27.598098 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e79a5ed-4ff0-49b6-b429-31894d0db9a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:27.605997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-4e79a5ed-4ff0-49b6-b429-31894d0db9a1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:07:27.605997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:27.605997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:27.605997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:27.605997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:27.605997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:27.605997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:27.605997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:27.605997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:27.605997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:27.605997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:27.605997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:27.605997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:27.605997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:27.605997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:27.605997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:27.606916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:27.606916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:27.606916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:27.606916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:27.606916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:27.606916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:27.606916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:27.606916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:27.606916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:27.606916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:27.606916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:27.606916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:27.606916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:27.606916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:27.606916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:27.606916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:27.607738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:27.607738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:27.607738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:27.607738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:27.607738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:27.607738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:27.607738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:27.607738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:27.607738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:27.607738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:27.607738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:07:27.607738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:27.607738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 158/309] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:27.608408 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79a5ed-4ff0-49b6-b429-31894d0db9a1 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:27.618812 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0385a-95bf-4626-94dd-04c2085ff698 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:27.655848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842fe89-b1d8-4567-a4f7-97b1a2f1725b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:27.656727 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842fe89-b1d8-4567-a4f7-97b1a2f1725b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:27.657009 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842fe89-b1d8-4567-a4f7-97b1a2f1725b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:27.657282 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842fe89-b1d8-4567-a4f7-97b1a2f1725b None admin] PATH_INFO: `/v3/users/da9e8f9fd6c54493bdca6f956c2eb20f` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:27.658483 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d842fe89-b1d8-4567-a4f7-97b1a2f1725b None admin] RBAC: Authorizing `identity:get_user(user_id=da9e8f9fd6c54493bdca6f956c2eb20f)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:27.659960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d842fe89-b1d8-4567-a4f7-97b1a2f1725b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:27.661616 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 152/310] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:27 2022] GET /identity/v3/users/da9e8f9fd6c54493bdca6f956c2eb20f => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:27.662382 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842fe89-b1d8-4567-a4f7-97b1a2f1725b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:27.721779 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79a5ed-4ff0-49b6-b429-31894d0db9a1 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:27.761557 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b307c-ddab-40a7-9ba6-4db8e7e4515d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:27.762498 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b307c-ddab-40a7-9ba6-4db8e7e4515d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:27.762782 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b307c-ddab-40a7-9ba6-4db8e7e4515d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:27.763090 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b307c-ddab-40a7-9ba6-4db8e7e4515d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:27.764363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b307c-ddab-40a7-9ba6-4db8e7e4515d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:27.765799 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b307c-ddab-40a7-9ba6-4db8e7e4515d None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:27.767655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 159/311] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:07:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:27.768222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b307c-ddab-40a7-9ba6-4db8e7e4515d None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:27.782137 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842fe89-b1d8-4567-a4f7-97b1a2f1725b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:27.802873 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:27.819526 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11a522-27b8-40ec-9248-ddb9c1b15770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:27.820590 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11a522-27b8-40ec-9248-ddb9c1b15770 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:27.821115 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11a522-27b8-40ec-9248-ddb9c1b15770 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:27.821589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11a522-27b8-40ec-9248-ddb9c1b15770 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:27.822517 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e11a522-27b8-40ec-9248-ddb9c1b15770 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:27.822996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e11a522-27b8-40ec-9248-ddb9c1b15770 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:27.824564 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e11a522-27b8-40ec-9248-ddb9c1b15770 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:27.834055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 153/312] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:07:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:27.835429 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11a522-27b8-40ec-9248-ddb9c1b15770 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:27.846082 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:27.894486 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b307c-ddab-40a7-9ba6-4db8e7e4515d None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:27.931754 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c16991-024f-42b7-b391-a96c07112052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:27.932410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c16991-024f-42b7-b391-a96c07112052 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:27.932694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c16991-024f-42b7-b391-a96c07112052 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:27.932969 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c16991-024f-42b7-b391-a96c07112052 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:27.948165 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11a522-27b8-40ec-9248-ddb9c1b15770 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:27.984398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4d459-5d32-42a6-93ea-c6d24341615f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:27.985184 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4d459-5d32-42a6-93ea-c6d24341615f None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:27.985473 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4d459-5d32-42a6-93ea-c6d24341615f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:27.985749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4d459-5d32-42a6-93ea-c6d24341615f None admin] PATH_INFO: `/v3/groups` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:27.986439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe4d459-5d32-42a6-93ea-c6d24341615f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:27.988049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe4d459-5d32-42a6-93ea-c6d24341615f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:28.002721 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c16991-024f-42b7-b391-a96c07112052 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:28.011513 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 154/313] 10.222.0.81 () {60 vars in 1134 bytes} [Tue Jun 7 14:07:27 2022] POST /identity/v3/groups => generated 229 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:07:28.011714 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c16991-024f-42b7-b391-a96c07112052 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:28.011793 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4d459-5d32-42a6-93ea-c6d24341615f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:28.073693 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:28.088719 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:28.147687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 160/314] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:07:28.153574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c16991-024f-42b7-b391-a96c07112052 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:28.156013 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4d459-5d32-42a6-93ea-c6d24341615f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:28.199725 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0cf6b-7261-42f8-8e09-c56b190cf94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:28.201118 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0cf6b-7261-42f8-8e09-c56b190cf94e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:28.201570 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0cf6b-7261-42f8-8e09-c56b190cf94e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:28.201982 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0cf6b-7261-42f8-8e09-c56b190cf94e None admin] PATH_INFO: `/v3/users/9d2bf0f62c074ba5adf2f2ed1db28541` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:28.218186 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb0cf6b-7261-42f8-8e09-c56b190cf94e None admin] RBAC: Authorizing `identity:get_user(user_id=9d2bf0f62c074ba5adf2f2ed1db28541)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:28.220020 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb0cf6b-7261-42f8-8e09-c56b190cf94e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:28.222315 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 155/315] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:28 2022] GET /identity/v3/users/9d2bf0f62c074ba5adf2f2ed1db28541 => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:28.228901 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0cf6b-7261-42f8-8e09-c56b190cf94e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:28.274253 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c16991-024f-42b7-b391-a96c07112052 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:28.313552 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58451d6-a6fe-46d6-a7ac-1a25920956f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:28.314838 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58451d6-a6fe-46d6-a7ac-1a25920956f4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:28.315112 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58451d6-a6fe-46d6-a7ac-1a25920956f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:28.315413 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58451d6-a6fe-46d6-a7ac-1a25920956f4 None admin] PATH_INFO: `/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:28.316990 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58451d6-a6fe-46d6-a7ac-1a25920956f4 None admin] RBAC: Authorizing `identity:get_project(project_id=c05a3987827c46a69dd1ab7b7f4b913b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:28.318783 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58451d6-a6fe-46d6-a7ac-1a25920956f4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:28.321268 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 161/316] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:28 2022] GET /identity/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:28.327174 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58451d6-a6fe-46d6-a7ac-1a25920956f4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:28.340387 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:28.344511 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0cf6b-7261-42f8-8e09-c56b190cf94e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:28.366350 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:28.386276 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4247da2-9661-4326-a5e4-f5656b77ce0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:28.387180 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4247da2-9661-4326-a5e4-f5656b77ce0c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:28.387686 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4247da2-9661-4326-a5e4-f5656b77ce0c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:28.388270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4247da2-9661-4326-a5e4-f5656b77ce0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:28.389417 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4247da2-9661-4326-a5e4-f5656b77ce0c None admin] RBAC: Adding query filter params (role.id=a1d21c4d26ef419298e001001f4655b7, user.id=9d2bf0f62c074ba5adf2f2ed1db28541, scope.system=all) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:28.389863 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4247da2-9661-4326-a5e4-f5656b77ce0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:28.391473 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4247da2-9661-4326-a5e4-f5656b77ce0c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:28.397679 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 156/317] 10.222.0.81 () {56 vars in 1308 bytes} [Tue Jun 7 14:07:28 2022] GET /identity/v3/role_assignments?user.id=9d2bf0f62c074ba5adf2f2ed1db28541&scope.system=all&role.id=a1d21c4d26ef419298e001001f4655b7 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:28.446856 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58451d6-a6fe-46d6-a7ac-1a25920956f4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:28.486579 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a304a-1ac6-4b3f-97e7-7109aeaecf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:28.487283 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a304a-1ac6-4b3f-97e7-7109aeaecf93 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:28.487577 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a304a-1ac6-4b3f-97e7-7109aeaecf93 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:28.487857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a304a-1ac6-4b3f-97e7-7109aeaecf93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:28.488784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a304a-1ac6-4b3f-97e7-7109aeaecf93 None admin] RBAC: Adding query filter params (role.id=a1d21c4d26ef419298e001001f4655b7, scope.project.id=c05a3987827c46a69dd1ab7b7f4b913b, user.id=da9e8f9fd6c54493bdca6f956c2eb20f) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:28.489501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a304a-1ac6-4b3f-97e7-7109aeaecf93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:28.490991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a304a-1ac6-4b3f-97e7-7109aeaecf93 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:28.499085 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 162/318] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:07:28 2022] GET /identity/v3/role_assignments?user.id=da9e8f9fd6c54493bdca6f956c2eb20f&scope.project.id=c05a3987827c46a69dd1ab7b7f4b913b&role.id=a1d21c4d26ef419298e001001f4655b7 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:29.679731 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd39717-76e1-4943-9ef9-fe0126ba8d76 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:29.680992 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd39717-76e1-4943-9ef9-fe0126ba8d76 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:29.680992 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd39717-76e1-4943-9ef9-fe0126ba8d76 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:29.682495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 157/319] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:29.687710 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe5227-e7ef-439f-981b-50a02cd3175c None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:29.688197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe5227-e7ef-439f-981b-50a02cd3175c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:29.688272 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe5227-e7ef-439f-981b-50a02cd3175c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:29.705114 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-94fe5227-e7ef-439f-981b-50a02cd3175c None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:29.706298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-94fe5227-e7ef-439f-981b-50a02cd3175c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:29.774157 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 163/320] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:29.797615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd39717-76e1-4943-9ef9-fe0126ba8d76 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:29.898239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd39717-76e1-4943-9ef9-fe0126ba8d76 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:29.932612 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a97d4b-6b52-46c4-8b0c-2746c3869df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:29.933332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a97d4b-6b52-46c4-8b0c-2746c3869df4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:29.933614 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a97d4b-6b52-46c4-8b0c-2746c3869df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:29.933884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a97d4b-6b52-46c4-8b0c-2746c3869df4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:29.943922 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a97d4b-6b52-46c4-8b0c-2746c3869df4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:29.952496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f3a97d4b-6b52-46c4-8b0c-2746c3869df4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:29.952496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:29.952496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:29.952496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:29.952496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:29.952496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:29.952496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:29.952496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:29.952496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:29.952496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:29.952496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:29.952496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:29.952496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:29.952496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:29.952496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:29.952496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:29.953415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:29.953415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:29.953415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:29.953415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:29.953415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:29.953415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:29.953415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:29.953415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:29.953415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:29.953415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:29.953415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:29.953415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:29.953415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:29.953415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:29.953415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:29.953415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:29.954202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:29.954202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:29.954202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:29.954202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:29.954202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:29.954202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:29.954202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:29.954202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:29.954202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:29.954202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:29.954202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:29.954202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:29.961468 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 158/321] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:29.961634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe5227-e7ef-439f-981b-50a02cd3175c None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:29.991172 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:30.069379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe5227-e7ef-439f-981b-50a02cd3175c None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:30.105675 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601afe35-a352-4bb8-84f7-40d185a90bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:30.106347 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601afe35-a352-4bb8-84f7-40d185a90bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:30.106630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601afe35-a352-4bb8-84f7-40d185a90bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:30.106902 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601afe35-a352-4bb8-84f7-40d185a90bb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:30.107679 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601afe35-a352-4bb8-84f7-40d185a90bb0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:30.107984 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601afe35-a352-4bb8-84f7-40d185a90bb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:30.109355 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601afe35-a352-4bb8-84f7-40d185a90bb0 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:30.119905 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 164/322] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:07:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:30.125582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a97d4b-6b52-46c4-8b0c-2746c3869df4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:30.153812 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:30.231111 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a97d4b-6b52-46c4-8b0c-2746c3869df4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:30.264793 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345eaa9f-5166-417c-99f6-5c4e544c39d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:30.265672 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345eaa9f-5166-417c-99f6-5c4e544c39d0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:30.265956 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345eaa9f-5166-417c-99f6-5c4e544c39d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:30.266231 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345eaa9f-5166-417c-99f6-5c4e544c39d0 None admin] PATH_INFO: `/v3/users/cad6af88a0844139adb20477a10e13c4` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:30.267476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345eaa9f-5166-417c-99f6-5c4e544c39d0 None admin] RBAC: Authorizing `identity:get_user(user_id=cad6af88a0844139adb20477a10e13c4)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:30.268911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345eaa9f-5166-417c-99f6-5c4e544c39d0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:30.270711 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 159/323] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:30 2022] GET /identity/v3/users/cad6af88a0844139adb20477a10e13c4 => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:30.276512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601afe35-a352-4bb8-84f7-40d185a90bb0 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:30.368414 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:30.382266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601afe35-a352-4bb8-84f7-40d185a90bb0 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:30.403721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0ef04-6d7b-46fc-8b48-a64e0e0a47a0 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:30.404062 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0ef04-6d7b-46fc-8b48-a64e0e0a47a0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:30.404282 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0ef04-6d7b-46fc-8b48-a64e0e0a47a0 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:30.407357 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 160/324] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:30.411825 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4edd6-14cc-408a-90ec-f0667f8b7933 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:30.412113 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4edd6-14cc-408a-90ec-f0667f8b7933 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:30.412392 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4edd6-14cc-408a-90ec-f0667f8b7933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:30.421969 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45266552-1231-4080-a70e-855f18032e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:30.422630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45266552-1231-4080-a70e-855f18032e02 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:30.422910 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45266552-1231-4080-a70e-855f18032e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:30.423184 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45266552-1231-4080-a70e-855f18032e02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:30.424049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45266552-1231-4080-a70e-855f18032e02 None admin] RBAC: Adding query filter params (role.id=ca91f64e7a46431c8aa4285c4620f216, user.id=cad6af88a0844139adb20477a10e13c4, scope.system=all) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:30.424360 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45266552-1231-4080-a70e-855f18032e02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:30.426215 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45266552-1231-4080-a70e-855f18032e02 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:30.430565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0aa4edd6-14cc-408a-90ec-f0667f8b7933 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:30.432192 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0aa4edd6-14cc-408a-90ec-f0667f8b7933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:30.439252 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 165/325] 10.222.0.81 () {56 vars in 1308 bytes} [Tue Jun 7 14:07:30 2022] GET /identity/v3/role_assignments?user.id=cad6af88a0844139adb20477a10e13c4&scope.system=all&role.id=ca91f64e7a46431c8aa4285c4620f216 => generated 510 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:30.505627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 161/326] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:07:30.530222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45266552-1231-4080-a70e-855f18032e02 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:30.643504 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45266552-1231-4080-a70e-855f18032e02 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:30.676908 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581390fd-f089-4272-bdaa-613f1bb5f601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:30.677699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581390fd-f089-4272-bdaa-613f1bb5f601 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:30.677986 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581390fd-f089-4272-bdaa-613f1bb5f601 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:30.678261 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581390fd-f089-4272-bdaa-613f1bb5f601 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:30.679468 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581390fd-f089-4272-bdaa-613f1bb5f601 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:30.681488 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581390fd-f089-4272-bdaa-613f1bb5f601 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:30.683442 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 166/327] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:07:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:30.689090 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4edd6-14cc-408a-90ec-f0667f8b7933 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:30.714575 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:30.735283 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0b91a-5d52-4ced-bfb3-4489a09cceb2 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:30.735799 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0b91a-5d52-4ced-bfb3-4489a09cceb2 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:30.735890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0b91a-5d52-4ced-bfb3-4489a09cceb2 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:30.737179 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 167/328] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:07:30.741991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1de69-cb4e-4dfe-b84b-5331da4fba19 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:30.742273 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1de69-cb4e-4dfe-b84b-5331da4fba19 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:30.743093 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1de69-cb4e-4dfe-b84b-5331da4fba19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:30.760187 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-e7d1de69-cb4e-4dfe-b84b-5331da4fba19 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:30.761281 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e7d1de69-cb4e-4dfe-b84b-5331da4fba19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:30.770293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:30.790986 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4edd6-14cc-408a-90ec-f0667f8b7933 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:30.826001 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b1d5f-b7d9-4d2d-beb7-59513c361e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:30.826676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b1d5f-b7d9-4d2d-beb7-59513c361e82 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:30.826963 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b1d5f-b7d9-4d2d-beb7-59513c361e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:30.827258 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b1d5f-b7d9-4d2d-beb7-59513c361e82 None admin] PATH_INFO: `/v3/groups` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:30.827349 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 168/329] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:07:30.828027 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324b1d5f-b7d9-4d2d-beb7-59513c361e82 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:30.829539 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324b1d5f-b7d9-4d2d-beb7-59513c361e82 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:30.842881 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 162/330] 10.222.0.81 () {60 vars in 1134 bytes} [Tue Jun 7 14:07:30 2022] POST /identity/v3/groups => generated 226 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:07:30.850475 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1de69-cb4e-4dfe-b84b-5331da4fba19 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:30.962091 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1de69-cb4e-4dfe-b84b-5331da4fba19 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:30.998622 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613e5e9-00df-4321-bf42-0a1263c83123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:30.999457 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613e5e9-00df-4321-bf42-0a1263c83123 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:30.999738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613e5e9-00df-4321-bf42-0a1263c83123 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:31.000008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613e5e9-00df-4321-bf42-0a1263c83123 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:31.012419 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b613e5e9-00df-4321-bf42-0a1263c83123 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:31.020277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-b613e5e9-00df-4321-bf42-0a1263c83123 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:07:31.020277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:31.020277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:31.020277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:31.020277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:31.020277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:31.020277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:31.020277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:31.020277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:31.020277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:31.020277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:31.020277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:31.020277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:31.020277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:31.020277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:31.020277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:31.021431 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:31.021431 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:31.021431 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:31.021431 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:31.021431 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:31.021431 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:31.021431 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:31.021431 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:31.021431 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:31.021431 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:31.021431 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:31.021431 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:31.021431 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:31.021431 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:31.021431 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:31.021431 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:31.022451 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:31.022451 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:31.022451 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:31.022451 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:31.022451 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:31.022451 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:31.022451 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:31.022451 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:31.022451 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:31.022451 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:31.022451 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:07:31.022451 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:31.022451 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 169/331] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:31.028118 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b1d5f-b7d9-4d2d-beb7-59513c361e82 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:31.111341 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:31.132642 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b1d5f-b7d9-4d2d-beb7-59513c361e82 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:31.173266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619072c-51cf-4529-8802-354fa9833a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:31.173266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619072c-51cf-4529-8802-354fa9833a1b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:31.173266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619072c-51cf-4529-8802-354fa9833a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:31.173266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619072c-51cf-4529-8802-354fa9833a1b None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:31.173624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c619072c-51cf-4529-8802-354fa9833a1b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:31.173845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c619072c-51cf-4529-8802-354fa9833a1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:31.175112 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c619072c-51cf-4529-8802-354fa9833a1b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:31.184951 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 163/332] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:07:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:31.190451 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613e5e9-00df-4321-bf42-0a1263c83123 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:31.194275 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:31.310959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613e5e9-00df-4321-bf42-0a1263c83123 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:31.346906 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f61201-751e-4b9e-9cde-3e4134413f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:31.347831 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f61201-751e-4b9e-9cde-3e4134413f1e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:31.348146 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f61201-751e-4b9e-9cde-3e4134413f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:31.348397 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f61201-751e-4b9e-9cde-3e4134413f1e None admin] PATH_INFO: `/v3/users/9d2bf0f62c074ba5adf2f2ed1db28541` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:31.349615 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f61201-751e-4b9e-9cde-3e4134413f1e None admin] RBAC: Authorizing `identity:get_user(user_id=9d2bf0f62c074ba5adf2f2ed1db28541)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:31.351068 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f61201-751e-4b9e-9cde-3e4134413f1e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:31.353675 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 170/333] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:31 2022] GET /identity/v3/users/9d2bf0f62c074ba5adf2f2ed1db28541 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:31.359321 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619072c-51cf-4529-8802-354fa9833a1b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:31.490114 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619072c-51cf-4529-8802-354fa9833a1b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:31.528164 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd237d5-c6c7-4898-83e2-30d00c1f83f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:31.529044 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd237d5-c6c7-4898-83e2-30d00c1f83f1 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:31.529330 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd237d5-c6c7-4898-83e2-30d00c1f83f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:31.529608 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd237d5-c6c7-4898-83e2-30d00c1f83f1 None admin] PATH_INFO: `/v3/system/users/9d2bf0f62c074ba5adf2f2ed1db28541/roles/a1d21c4d26ef419298e001001f4655b7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:31.531186 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd237d5-c6c7-4898-83e2-30d00c1f83f1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9d2bf0f62c074ba5adf2f2ed1db28541, role_id=a1d21c4d26ef419298e001001f4655b7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:31.532620 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd237d5-c6c7-4898-83e2-30d00c1f83f1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:31.542209 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 164/334] 10.222.0.81 () {58 vars in 1334 bytes} [Tue Jun 7 14:07:31 2022] PUT /identity/v3/system/users/9d2bf0f62c074ba5adf2f2ed1db28541/roles/a1d21c4d26ef419298e001001f4655b7 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:07:33.450299 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea27fd-66fc-4038-8f82-b0ad0351fd8b None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:33.451286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea27fd-66fc-4038-8f82-b0ad0351fd8b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:33.451782 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea27fd-66fc-4038-8f82-b0ad0351fd8b None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:33.453647 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 171/335] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:33.458885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e97ed3-0501-4758-aedf-bd65f0a0634e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:33.459776 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e97ed3-0501-4758-aedf-bd65f0a0634e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:33.460257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e97ed3-0501-4758-aedf-bd65f0a0634e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:33.481215 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-96e97ed3-0501-4758-aedf-bd65f0a0634e None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:33.482801 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-96e97ed3-0501-4758-aedf-bd65f0a0634e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:33.485511 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:33.571585 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 165/336] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:33.609719 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea27fd-66fc-4038-8f82-b0ad0351fd8b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:33.694923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:33.737274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea27fd-66fc-4038-8f82-b0ad0351fd8b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:33.781624 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1787212-43ab-463c-85ec-b2bae1831704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:33.782443 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1787212-43ab-463c-85ec-b2bae1831704 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:33.782730 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1787212-43ab-463c-85ec-b2bae1831704 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:33.783007 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1787212-43ab-463c-85ec-b2bae1831704 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:33.794593 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1787212-43ab-463c-85ec-b2bae1831704 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:33.802447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-b1787212-43ab-463c-85ec-b2bae1831704 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:33.802447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:33.802447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:33.802447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:33.802447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:33.802447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:33.802447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:33.802447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:33.802447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:33.802447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:33.802447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:33.802447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:33.802447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:33.802447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:33.802447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:33.802447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:33.803400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:33.803400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:33.803400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:33.803400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:33.803400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:33.803400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:33.803400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:33.803400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:33.803400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:33.803400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:33.803400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:33.803400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:33.803400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:33.803400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:33.803400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:33.803400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:33.804214 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:33.804214 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:33.804214 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:33.804214 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:33.804214 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:33.804214 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:33.804214 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:33.804214 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:33.804214 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:33.804214 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:33.804214 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:33.804214 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:33.804214 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 172/337] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:33.810443 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e97ed3-0501-4758-aedf-bd65f0a0634e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:33.956527 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e97ed3-0501-4758-aedf-bd65f0a0634e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:34.001941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883bbcf-1e7d-4f8e-8cbb-ba948713c81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:34.002928 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883bbcf-1e7d-4f8e-8cbb-ba948713c81a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:34.003444 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883bbcf-1e7d-4f8e-8cbb-ba948713c81a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:34.003919 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883bbcf-1e7d-4f8e-8cbb-ba948713c81a None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:34.004861 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4883bbcf-1e7d-4f8e-8cbb-ba948713c81a None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:34.005372 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4883bbcf-1e7d-4f8e-8cbb-ba948713c81a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:34.006939 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4883bbcf-1e7d-4f8e-8cbb-ba948713c81a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:34.024892 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:34.025220 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 166/338] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:07:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:34.031671 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1787212-43ab-463c-85ec-b2bae1831704 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:34.185340 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1787212-43ab-463c-85ec-b2bae1831704 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:34.227661 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:34.231880 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc39c2-2f91-40fc-88bf-952899e41bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:34.232864 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc39c2-2f91-40fc-88bf-952899e41bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:34.233150 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc39c2-2f91-40fc-88bf-952899e41bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:34.233427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc39c2-2f91-40fc-88bf-952899e41bb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:34.270974 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e14aa4-6ebb-46fe-a023-ee014c3b86fb None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:34.271308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e14aa4-6ebb-46fe-a023-ee014c3b86fb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:34.271474 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e14aa4-6ebb-46fe-a023-ee014c3b86fb None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:34.273106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 167/339] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:34.278413 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83ec42-c4d8-4c97-9f92-302ff2241738 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:34.278910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83ec42-c4d8-4c97-9f92-302ff2241738 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:34.279378 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83ec42-c4d8-4c97-9f92-302ff2241738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:34.310968 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc39c2-2f91-40fc-88bf-952899e41bb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:34.312968 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc39c2-2f91-40fc-88bf-952899e41bb9 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:34.327837 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ab83ec42-c4d8-4c97-9f92-302ff2241738 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:34.329301 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ab83ec42-c4d8-4c97-9f92-302ff2241738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:34.465715 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 168/340] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:07:34.467813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10c7ad-4e97-44aa-8b20-f2cac966f663 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:34.469565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10c7ad-4e97-44aa-8b20-f2cac966f663 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:34.470018 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10c7ad-4e97-44aa-8b20-f2cac966f663 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:34.472674 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 169/341] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:34.478979 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11274519-ea46-45a9-8c24-5d89f66067c8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:34.479505 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11274519-ea46-45a9-8c24-5d89f66067c8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:34.479942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11274519-ea46-45a9-8c24-5d89f66067c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:34.504976 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 173/342] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:07:34.511099 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc39c2-2f91-40fc-88bf-952899e41bb9 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:34.511570 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-11274519-ea46-45a9-8c24-5d89f66067c8 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:34.513124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-11274519-ea46-45a9-8c24-5d89f66067c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:34.551665 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:34.623640 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc39c2-2f91-40fc-88bf-952899e41bb9 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:34.645407 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 170/343] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:07:34.647407 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11274519-ea46-45a9-8c24-5d89f66067c8 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:34.690707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df2e5d3-f706-4cd4-9b74-c48ec8e2d20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:34.691814 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2e5d3-f706-4cd4-9b74-c48ec8e2d20b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:34.692242 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2e5d3-f706-4cd4-9b74-c48ec8e2d20b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:34.692689 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2e5d3-f706-4cd4-9b74-c48ec8e2d20b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:34.713554 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df2e5d3-f706-4cd4-9b74-c48ec8e2d20b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:34.721859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-3df2e5d3-f706-4cd4-9b74-c48ec8e2d20b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:07:34.721859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:34.721859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:34.721859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:34.721859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:34.721859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:34.721859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:34.721859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:34.721859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:34.721859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:34.721859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:34.721859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:34.721859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:34.721859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:34.721859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:34.721859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:34.722794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:34.722794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:34.722794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:34.722794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:34.722794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:34.722794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:34.722794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:34.722794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:34.722794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:34.722794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:34.722794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:34.722794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:34.722794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:34.722794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:34.722794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:34.722794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:34.723630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:34.723630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:34.723630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:34.723630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:34.723630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:34.723630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:34.723630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:34.723630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:34.723630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:34.723630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:34.723630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:07:34.723630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:34.723630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 174/344] 10.222.0.81 () {56 vars in 1114 bytes} [Tue Jun 7 14:07:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:07:34.732247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df2e5d3-f706-4cd4-9b74-c48ec8e2d20b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:34.744324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:34.835287 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11274519-ea46-45a9-8c24-5d89f66067c8 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:34.891729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:34.897326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df2e5d3-f706-4cd4-9b74-c48ec8e2d20b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:34.917902 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858a51a-5b9c-4551-bceb-697a9528de3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:34.918810 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858a51a-5b9c-4551-bceb-697a9528de3d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:34.919089 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858a51a-5b9c-4551-bceb-697a9528de3d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:34.919403 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858a51a-5b9c-4551-bceb-697a9528de3d None admin] PATH_INFO: `/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:34.924461 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2858a51a-5b9c-4551-bceb-697a9528de3d None admin] RBAC: Authorizing `identity:get_project(project_id=a85ae30482264d7ba1c84ab0ca46d4e0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:34.925962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2858a51a-5b9c-4551-bceb-697a9528de3d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:34.927802 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 171/345] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:34 2022] GET /identity/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0 => generated 298 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:34.928576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858a51a-5b9c-4551-bceb-697a9528de3d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:34.969528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d8c290-6b9d-471f-be04-7284b56f03b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:34.970335 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8c290-6b9d-471f-be04-7284b56f03b4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:34.970618 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8c290-6b9d-471f-be04-7284b56f03b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:34.970891 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8c290-6b9d-471f-be04-7284b56f03b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:34.993169 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d8c290-6b9d-471f-be04-7284b56f03b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:35.011427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-88d8c290-6b9d-471f-be04-7284b56f03b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:35.011427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:35.011427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:35.011427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:35.011427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:35.011427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:35.011427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:35.011427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:35.011427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:35.011427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:35.011427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:35.011427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:35.011427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:35.011427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:35.011427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:35.011427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:35.012403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:35.012403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:35.012403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:35.012403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:35.012403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:35.012403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:35.012403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:35.012403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:35.012403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:35.012403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:35.012403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:35.012403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:35.012403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:35.012403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:35.012403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:35.012403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:35.013293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:35.013293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:35.013293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:35.013293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:35.013293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:35.013293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:35.013293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:35.013293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:35.013293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:35.013293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:35.013293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:35.013293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:35.013293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 175/346] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:35.013959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d8c290-6b9d-471f-be04-7284b56f03b4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:35.134817 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858a51a-5b9c-4551-bceb-697a9528de3d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:35.191062 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:35.200723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfd738-c4bf-46bc-aa90-9e7c7035d82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:35.201740 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfd738-c4bf-46bc-aa90-9e7c7035d82d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:35.202229 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfd738-c4bf-46bc-aa90-9e7c7035d82d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:35.202707 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfd738-c4bf-46bc-aa90-9e7c7035d82d None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:35.203729 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfd738-c4bf-46bc-aa90-9e7c7035d82d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:35.203845 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d8c290-6b9d-471f-be04-7284b56f03b4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:35.204354 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfd738-c4bf-46bc-aa90-9e7c7035d82d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:35.205978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfd738-c4bf-46bc-aa90-9e7c7035d82d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:35.216068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 172/347] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:07:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:35.217282 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfd738-c4bf-46bc-aa90-9e7c7035d82d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:35.252236 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8394272-49e6-4c97-a032-8bd412644550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:35.253357 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8394272-49e6-4c97-a032-8bd412644550 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:35.253778 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8394272-49e6-4c97-a032-8bd412644550 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:35.254213 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8394272-49e6-4c97-a032-8bd412644550 None admin] PATH_INFO: `/v3/groups/14581f2851d6447b8f3c0df3db37ca66` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:35.261727 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8394272-49e6-4c97-a032-8bd412644550 None admin] RBAC: Authorizing `identity:get_group(group_id=14581f2851d6447b8f3c0df3db37ca66)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:35.263566 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8394272-49e6-4c97-a032-8bd412644550 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:35.270370 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 176/348] 10.222.0.81 () {56 vars in 1180 bytes} [Tue Jun 7 14:07:35 2022] GET /identity/v3/groups/14581f2851d6447b8f3c0df3db37ca66 => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:35.272155 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8b93b-1ffe-4c24-b9db-c2f2b960f5c4 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:35.272597 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8b93b-1ffe-4c24-b9db-c2f2b960f5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:35.273032 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8b93b-1ffe-4c24-b9db-c2f2b960f5c4 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:35.275376 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 177/349] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:35.276360 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8b93b-1ffe-4c24-b9db-c2f2b960f5c4 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:35.298029 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:35.346858 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfd738-c4bf-46bc-aa90-9e7c7035d82d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:35.390813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe7b07-a5c8-42ba-baad-cb867bf17c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:35.391759 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe7b07-a5c8-42ba-baad-cb867bf17c85 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:35.392194 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe7b07-a5c8-42ba-baad-cb867bf17c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:35.392615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe7b07-a5c8-42ba-baad-cb867bf17c85 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:35.393507 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe7b07-a5c8-42ba-baad-cb867bf17c85 None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:35.393950 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe7b07-a5c8-42ba-baad-cb867bf17c85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:35.395503 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe7b07-a5c8-42ba-baad-cb867bf17c85 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:35.405507 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 173/350] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:07:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:35.406633 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe7b07-a5c8-42ba-baad-cb867bf17c85 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:35.430064 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8b93b-1ffe-4c24-b9db-c2f2b960f5c4 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:35.472380 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:35.482393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb99d85-05f1-4418-a53d-44c11ad42ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:35.483367 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb99d85-05f1-4418-a53d-44c11ad42ffc None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:35.483804 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb99d85-05f1-4418-a53d-44c11ad42ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:35.484232 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb99d85-05f1-4418-a53d-44c11ad42ffc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:35.529056 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe7b07-a5c8-42ba-baad-cb867bf17c85 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:35.578431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b28e87-cf5a-418a-952f-e72f5fe16236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:35.579024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b28e87-cf5a-418a-952f-e72f5fe16236 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:35.579369 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b28e87-cf5a-418a-952f-e72f5fe16236 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:35.579649 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b28e87-cf5a-418a-952f-e72f5fe16236 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:35.580499 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b28e87-cf5a-418a-952f-e72f5fe16236 None admin] RBAC: Adding query filter params (group.id=14581f2851d6447b8f3c0df3db37ca66, role.id=ca91f64e7a46431c8aa4285c4620f216, scope.project.id=a85ae30482264d7ba1c84ab0ca46d4e0) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:35.580814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b28e87-cf5a-418a-952f-e72f5fe16236 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:35.582209 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb99d85-05f1-4418-a53d-44c11ad42ffc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:35.582633 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b28e87-cf5a-418a-952f-e72f5fe16236 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:35.583704 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb99d85-05f1-4418-a53d-44c11ad42ffc None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:35.589027 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 174/351] 10.222.0.81 () {56 vars in 1376 bytes} [Tue Jun 7 14:07:35 2022] GET /identity/v3/role_assignments?group.id=14581f2851d6447b8f3c0df3db37ca66&scope.project.id=a85ae30482264d7ba1c84ab0ca46d4e0&role.id=ca91f64e7a46431c8aa4285c4620f216 => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:35.595387 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6deea-b21f-4047-be7a-8e5891252459 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:35.595387 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6deea-b21f-4047-be7a-8e5891252459 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:35.595387 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6deea-b21f-4047-be7a-8e5891252459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:35.612733 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d5c6deea-b21f-4047-be7a-8e5891252459 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:35.613976 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d5c6deea-b21f-4047-be7a-8e5891252459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:35.615542 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:35.694591 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 175/352] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:35.695858 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19326ec-f44b-40ff-ad5e-4c7546fc7d36 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:35.696162 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19326ec-f44b-40ff-ad5e-4c7546fc7d36 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:35.696439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19326ec-f44b-40ff-ad5e-4c7546fc7d36 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:35.698168 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 176/353] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:35.698940 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19326ec-f44b-40ff-ad5e-4c7546fc7d36 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:35.719517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 178/354] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:07:35.727452 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a8944-cc6d-489f-ab8e-483088fb0312 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:35.728149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a8944-cc6d-489f-ab8e-483088fb0312 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:35.728637 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a8944-cc6d-489f-ab8e-483088fb0312 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:35.730575 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 179/355] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:35 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:35.732178 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2c13a-a6f2-4a55-abd5-2811bece1c19 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:35.732642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2c13a-a6f2-4a55-abd5-2811bece1c19 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:35.733080 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2c13a-a6f2-4a55-abd5-2811bece1c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:35.754389 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-09e2c13a-a6f2-4a55-abd5-2811bece1c19 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:35.755976 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-09e2c13a-a6f2-4a55-abd5-2811bece1c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:35.761980 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:35.837254 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19326ec-f44b-40ff-ad5e-4c7546fc7d36 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:35.847980 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 180/356] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:35.848830 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2c13a-a6f2-4a55-abd5-2811bece1c19 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:35.879470 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1ce85-d8a7-4c53-bc36-5b43f300035c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:35.880443 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1ce85-d8a7-4c53-bc36-5b43f300035c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:35.880927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1ce85-d8a7-4c53-bc36-5b43f300035c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:35.881422 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1ce85-d8a7-4c53-bc36-5b43f300035c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:35.928865 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:35.973491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b1ce85-d8a7-4c53-bc36-5b43f300035c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:35.975085 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b1ce85-d8a7-4c53-bc36-5b43f300035c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:36.004446 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2c13a-a6f2-4a55-abd5-2811bece1c19 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:36.036497 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:36.051183 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8828db2-d100-415d-9dfb-0067953dc318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:36.052080 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8828db2-d100-415d-9dfb-0067953dc318 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:36.052327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8828db2-d100-415d-9dfb-0067953dc318 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:36.052598 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8828db2-d100-415d-9dfb-0067953dc318 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:36.065857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8828db2-d100-415d-9dfb-0067953dc318 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:36.073687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-e8828db2-d100-415d-9dfb-0067953dc318 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:07:36.073687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:36.073687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:36.073687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:36.073687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:36.073687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:36.073687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:36.073687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:36.073687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:36.073687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:36.073687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:36.073687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:36.073687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:36.073687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:36.073687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:36.073687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:36.074658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:36.074658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:36.074658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:36.074658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:36.074658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:36.074658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:36.074658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:36.074658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:36.074658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:36.074658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:36.074658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:36.074658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:36.074658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:36.074658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:36.074658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:36.074658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:36.075685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:36.075685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:36.075685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:36.075685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:36.075685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:36.075685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:36.075685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:36.075685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:36.075685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:36.075685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:36.075685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:07:36.075685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:36.075685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 181/357] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:36.075685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8828db2-d100-415d-9dfb-0067953dc318 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:36.109718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 177/358] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:07:36.110179 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae41a46-83d5-4afc-b2e5-caaf4d4edb88 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:36.110470 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae41a46-83d5-4afc-b2e5-caaf4d4edb88 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:36.110742 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae41a46-83d5-4afc-b2e5-caaf4d4edb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:36.132536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0ae41a46-83d5-4afc-b2e5-caaf4d4edb88 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:36.134512 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0ae41a46-83d5-4afc-b2e5-caaf4d4edb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:36.220016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8828db2-d100-415d-9dfb-0067953dc318 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:36.220742 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 178/359] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:36.221807 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae41a46-83d5-4afc-b2e5-caaf4d4edb88 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:36.229058 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:36.261309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801f4a7-f0fd-4686-89db-211fde6a7d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:36.262441 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801f4a7-f0fd-4686-89db-211fde6a7d97 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:36.262873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801f4a7-f0fd-4686-89db-211fde6a7d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:36.263465 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801f4a7-f0fd-4686-89db-211fde6a7d97 None admin] PATH_INFO: `/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:36.265601 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801f4a7-f0fd-4686-89db-211fde6a7d97 None admin] RBAC: Authorizing `identity:get_project(project_id=a85ae30482264d7ba1c84ab0ca46d4e0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:36.267191 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801f4a7-f0fd-4686-89db-211fde6a7d97 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:36.269725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 182/360] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:36 2022] GET /identity/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:36.270630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801f4a7-f0fd-4686-89db-211fde6a7d97 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:36.315071 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:36.343716 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae41a46-83d5-4afc-b2e5-caaf4d4edb88 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:36.387164 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ebc060-4f21-4438-840d-e7ae691ef30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:36.388009 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebc060-4f21-4438-840d-e7ae691ef30c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:36.388295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebc060-4f21-4438-840d-e7ae691ef30c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:36.388571 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebc060-4f21-4438-840d-e7ae691ef30c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:36.395538 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801f4a7-f0fd-4686-89db-211fde6a7d97 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:36.399380 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ebc060-4f21-4438-840d-e7ae691ef30c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:36.409598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-84ebc060-4f21-4438-840d-e7ae691ef30c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:07:36.409598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:36.409598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:36.409598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:36.409598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:36.409598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:36.409598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:36.409598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:36.409598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:36.409598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:36.409598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:36.409598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:36.409598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:36.409598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:36.409598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:36.409598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:36.410544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:36.410544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:36.410544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:36.410544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:36.410544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:36.410544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:36.410544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:36.410544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:36.410544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:36.410544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:36.410544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:36.410544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:36.410544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:36.410544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:36.410544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:36.410544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:36.411411 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:36.411411 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:36.411411 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:36.411411 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:36.411411 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:36.411411 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:36.411411 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:36.411411 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:36.411411 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:36.411411 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:36.411411 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:07:36.411411 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:36.415929 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 179/361] 10.222.0.81 () {56 vars in 1114 bytes} [Tue Jun 7 14:07:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:36.417135 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ebc060-4f21-4438-840d-e7ae691ef30c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:36.442617 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2dde61-15d7-4603-b3a9-e75c301b2306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:36.443624 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dde61-15d7-4603-b3a9-e75c301b2306 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:36.443915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dde61-15d7-4603-b3a9-e75c301b2306 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:36.445841 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dde61-15d7-4603-b3a9-e75c301b2306 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:36.445841 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2dde61-15d7-4603-b3a9-e75c301b2306 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:36.445841 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2dde61-15d7-4603-b3a9-e75c301b2306 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:36.447707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2dde61-15d7-4603-b3a9-e75c301b2306 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:36.457886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 183/362] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:07:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:36.458573 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2dde61-15d7-4603-b3a9-e75c301b2306 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:36.511193 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:36.559045 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ebc060-4f21-4438-840d-e7ae691ef30c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:36.598797 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a878d66-118e-4555-9095-f712fea04007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:36.599724 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a878d66-118e-4555-9095-f712fea04007 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:36.600012 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a878d66-118e-4555-9095-f712fea04007 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:36.600288 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a878d66-118e-4555-9095-f712fea04007 None admin] PATH_INFO: `/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:36.603269 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a878d66-118e-4555-9095-f712fea04007 None admin] RBAC: Authorizing `identity:get_project(project_id=c05a3987827c46a69dd1ab7b7f4b913b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:36.604831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a878d66-118e-4555-9095-f712fea04007 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:36.606579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 180/363] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:36 2022] GET /identity/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:36.607017 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2dde61-15d7-4603-b3a9-e75c301b2306 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:36.608318 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a878d66-118e-4555-9095-f712fea04007 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:36.614008 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:36.654220 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc6737-ab4b-48f3-9a97-df2ba5c9cc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:36.655026 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc6737-ab4b-48f3-9a97-df2ba5c9cc2b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:36.655348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc6737-ab4b-48f3-9a97-df2ba5c9cc2b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:36.655628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc6737-ab4b-48f3-9a97-df2ba5c9cc2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:36.673870 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfc6737-ab4b-48f3-9a97-df2ba5c9cc2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:36.682945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-abfc6737-ab4b-48f3-9a97-df2ba5c9cc2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:07:36.682945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:36.682945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:36.682945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:36.682945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:36.682945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:36.682945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:36.682945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:36.682945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:36.682945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:36.682945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:36.682945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:36.682945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:36.682945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:36.682945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:36.682945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:36.683890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:36.683890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:36.683890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:36.683890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:36.683890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:36.683890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:36.683890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:36.683890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:36.683890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:36.683890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:36.683890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:36.683890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:36.683890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:36.683890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:36.683890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:36.683890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:36.684833 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:36.684833 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:36.684833 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:36.684833 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:36.684833 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:36.684833 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:36.684833 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:36.684833 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:36.684833 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:36.684833 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:36.684833 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:07:36.684833 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:36.686858 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 184/364] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:07:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:36.688034 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc6737-ab4b-48f3-9a97-df2ba5c9cc2b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:36.726484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a878d66-118e-4555-9095-f712fea04007 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:36.776605 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74702d15-3116-4678-b0b1-98b6bcd4b094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:36.777567 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74702d15-3116-4678-b0b1-98b6bcd4b094 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:36.777950 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74702d15-3116-4678-b0b1-98b6bcd4b094 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:36.778323 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74702d15-3116-4678-b0b1-98b6bcd4b094 None admin] PATH_INFO: `/v3/groups/14581f2851d6447b8f3c0df3db37ca66` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:36.780546 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74702d15-3116-4678-b0b1-98b6bcd4b094 None admin] RBAC: Authorizing `identity:get_group(group_id=14581f2851d6447b8f3c0df3db37ca66)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:36.782111 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74702d15-3116-4678-b0b1-98b6bcd4b094 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:36.784159 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 181/365] 10.222.0.81 () {56 vars in 1180 bytes} [Tue Jun 7 14:07:36 2022] GET /identity/v3/groups/14581f2851d6447b8f3c0df3db37ca66 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:36.784829 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74702d15-3116-4678-b0b1-98b6bcd4b094 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:36.804278 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:36.811058 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc6737-ab4b-48f3-9a97-df2ba5c9cc2b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:36.856892 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1071b6-a601-49cf-9a4c-78fc426f11d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:36.857577 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1071b6-a601-49cf-9a4c-78fc426f11d0 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:36.857869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1071b6-a601-49cf-9a4c-78fc426f11d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:36.858151 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1071b6-a601-49cf-9a4c-78fc426f11d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:36.858916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1071b6-a601-49cf-9a4c-78fc426f11d0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:36.859257 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1071b6-a601-49cf-9a4c-78fc426f11d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:36.861402 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1071b6-a601-49cf-9a4c-78fc426f11d0 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:36.871333 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 185/366] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:07:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:36.872442 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1071b6-a601-49cf-9a4c-78fc426f11d0 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:36.895349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:36.936569 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74702d15-3116-4678-b0b1-98b6bcd4b094 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:36.986262 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69eb7b3-e542-49c1-ab10-bc22ccfc8b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:36.987289 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69eb7b3-e542-49c1-ab10-bc22ccfc8b3d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:36.987643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69eb7b3-e542-49c1-ab10-bc22ccfc8b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:36.987996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69eb7b3-e542-49c1-ab10-bc22ccfc8b3d None admin] PATH_INFO: `/v3/users/9d2bf0f62c074ba5adf2f2ed1db28541` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:36.989457 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69eb7b3-e542-49c1-ab10-bc22ccfc8b3d None admin] RBAC: Authorizing `identity:get_user(user_id=9d2bf0f62c074ba5adf2f2ed1db28541)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:36.991070 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69eb7b3-e542-49c1-ab10-bc22ccfc8b3d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:36.993164 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 182/367] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:07:36 2022] GET /identity/v3/users/9d2bf0f62c074ba5adf2f2ed1db28541 => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:36.993954 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69eb7b3-e542-49c1-ab10-bc22ccfc8b3d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:37.015140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1071b6-a601-49cf-9a4c-78fc426f11d0 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:37.058619 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32d1df-7183-4519-9220-d12c994cd351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:37.059447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32d1df-7183-4519-9220-d12c994cd351 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:37.059730 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32d1df-7183-4519-9220-d12c994cd351 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:37.060007 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32d1df-7183-4519-9220-d12c994cd351 None admin] PATH_INFO: `/v3/groups/14581f2851d6447b8f3c0df3db37ca66` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:37.061198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32d1df-7183-4519-9220-d12c994cd351 None admin] RBAC: Authorizing `identity:get_group(group_id=14581f2851d6447b8f3c0df3db37ca66)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:37.062636 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32d1df-7183-4519-9220-d12c994cd351 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:37.064711 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 186/368] 10.222.0.81 () {56 vars in 1180 bytes} [Tue Jun 7 14:07:36 2022] GET /identity/v3/groups/14581f2851d6447b8f3c0df3db37ca66 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:37.065311 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32d1df-7183-4519-9220-d12c994cd351 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:37.102825 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:37.137511 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69eb7b3-e542-49c1-ab10-bc22ccfc8b3d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:37.189395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab4fbc-e9b2-40df-b2a8-b59472c502ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:37.190191 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab4fbc-e9b2-40df-b2a8-b59472c502ee None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:37.190544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab4fbc-e9b2-40df-b2a8-b59472c502ee None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:37.190877 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab4fbc-e9b2-40df-b2a8-b59472c502ee None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:37.191667 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab4fbc-e9b2-40df-b2a8-b59472c502ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:37.191969 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab4fbc-e9b2-40df-b2a8-b59472c502ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:37.193500 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab4fbc-e9b2-40df-b2a8-b59472c502ee None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:37.197781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32d1df-7183-4519-9220-d12c994cd351 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:37.207062 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 183/369] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:37.207062 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab4fbc-e9b2-40df-b2a8-b59472c502ee None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:37.212667 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:37.236468 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1ce2c-af10-42fb-bf89-ae075da23868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:37.237139 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1ce2c-af10-42fb-bf89-ae075da23868 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:37.237486 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1ce2c-af10-42fb-bf89-ae075da23868 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:37.237763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1ce2c-af10-42fb-bf89-ae075da23868 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:37.238621 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1ce2c-af10-42fb-bf89-ae075da23868 None admin] RBAC: Adding query filter params (group.id=14581f2851d6447b8f3c0df3db37ca66, role.id=6d02db1fe03b447c9e519f4f2aa16574, scope.project.id=a85ae30482264d7ba1c84ab0ca46d4e0) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:37.238938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1ce2c-af10-42fb-bf89-ae075da23868 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:37.240413 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1ce2c-af10-42fb-bf89-ae075da23868 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:37.252626 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 187/370] 10.222.0.81 () {56 vars in 1376 bytes} [Tue Jun 7 14:07:37 2022] GET /identity/v3/role_assignments?group.id=14581f2851d6447b8f3c0df3db37ca66&scope.project.id=a85ae30482264d7ba1c84ab0ca46d4e0&role.id=6d02db1fe03b447c9e519f4f2aa16574 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:37.255389 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1ce2c-af10-42fb-bf89-ae075da23868 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:37.327923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab4fbc-e9b2-40df-b2a8-b59472c502ee None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:37.367667 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82bb0d-d7e1-47ce-9a7b-263a5e1e059c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:37.369005 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82bb0d-d7e1-47ce-9a7b-263a5e1e059c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:37.369506 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82bb0d-d7e1-47ce-9a7b-263a5e1e059c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:37.369987 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82bb0d-d7e1-47ce-9a7b-263a5e1e059c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:37.373256 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:37.386010 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1ce2c-af10-42fb-bf89-ae075da23868 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:37.426943 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4f60b-f8fe-4993-93fa-b5a97f4eeff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:37.427731 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4f60b-f8fe-4993-93fa-b5a97f4eeff7 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:37.428022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4f60b-f8fe-4993-93fa-b5a97f4eeff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:37.428303 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4f60b-f8fe-4993-93fa-b5a97f4eeff7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:37.429261 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4f60b-f8fe-4993-93fa-b5a97f4eeff7 None admin] RBAC: Adding query filter params (role.id=a1d21c4d26ef419298e001001f4655b7, user.id=9d2bf0f62c074ba5adf2f2ed1db28541, scope.system=all) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:37.429571 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4f60b-f8fe-4993-93fa-b5a97f4eeff7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:37.431435 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4f60b-f8fe-4993-93fa-b5a97f4eeff7 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:37.445786 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 188/371] 10.222.0.81 () {56 vars in 1308 bytes} [Tue Jun 7 14:07:37 2022] GET /identity/v3/role_assignments?user.id=9d2bf0f62c074ba5adf2f2ed1db28541&scope.system=all&role.id=a1d21c4d26ef419298e001001f4655b7 => generated 510 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:37.446605 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4f60b-f8fe-4993-93fa-b5a97f4eeff7 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:37.453192 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e82bb0d-d7e1-47ce-9a7b-263a5e1e059c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:37.455036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e82bb0d-d7e1-47ce-9a7b-263a5e1e059c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:37.465437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:37.625198 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 184/372] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:07:37.627040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82bb0d-d7e1-47ce-9a7b-263a5e1e059c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:37.679814 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4f60b-f8fe-4993-93fa-b5a97f4eeff7 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:37.724309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13a394-1eb3-48cb-bbd6-32474ce6c8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:37.725313 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13a394-1eb3-48cb-bbd6-32474ce6c8dd None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:37.727238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13a394-1eb3-48cb-bbd6-32474ce6c8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:37.727782 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13a394-1eb3-48cb-bbd6-32474ce6c8dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:37.728906 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13a394-1eb3-48cb-bbd6-32474ce6c8dd None admin] RBAC: Adding query filter params (group.id=14581f2851d6447b8f3c0df3db37ca66, role.id=ca91f64e7a46431c8aa4285c4620f216, scope.project.id=c05a3987827c46a69dd1ab7b7f4b913b) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:37.729439 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13a394-1eb3-48cb-bbd6-32474ce6c8dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:37.731098 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13a394-1eb3-48cb-bbd6-32474ce6c8dd None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:37.737445 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 189/373] 10.222.0.81 () {56 vars in 1376 bytes} [Tue Jun 7 14:07:37 2022] GET /identity/v3/role_assignments?group.id=14581f2851d6447b8f3c0df3db37ca66&scope.project.id=c05a3987827c46a69dd1ab7b7f4b913b&role.id=ca91f64e7a46431c8aa4285c4620f216 => generated 260 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:37.744594 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13a394-1eb3-48cb-bbd6-32474ce6c8dd None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:37.760489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:37.778821 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82bb0d-d7e1-47ce-9a7b-263a5e1e059c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:37.780895 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:37.823060 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13185846-b147-406d-9f19-094c82fd68c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:37.824127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13185846-b147-406d-9f19-094c82fd68c3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:37.824631 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13185846-b147-406d-9f19-094c82fd68c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:37.825101 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13185846-b147-406d-9f19-094c82fd68c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:37.879019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13a394-1eb3-48cb-bbd6-32474ce6c8dd None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:37.908051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13185846-b147-406d-9f19-094c82fd68c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:37.910043 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13185846-b147-406d-9f19-094c82fd68c3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:37.917528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ed304-d075-4af1-83e3-f9237af54463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:37.918418 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ed304-d075-4af1-83e3-f9237af54463 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:37.918703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ed304-d075-4af1-83e3-f9237af54463 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:37.918996 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ed304-d075-4af1-83e3-f9237af54463 None admin] PATH_INFO: `/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:37.920293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ed304-d075-4af1-83e3-f9237af54463 None admin] RBAC: Authorizing `identity:get_project(project_id=c05a3987827c46a69dd1ab7b7f4b913b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:37.921764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ed304-d075-4af1-83e3-f9237af54463 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:37.923756 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 190/374] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:37 2022] GET /identity/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:37.929542 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ed304-d075-4af1-83e3-f9237af54463 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:38.018620 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:38.032536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 185/375] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:07:38.046772 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ed304-d075-4af1-83e3-f9237af54463 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:38.055505 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13185846-b147-406d-9f19-094c82fd68c3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:38.058130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:38.088460 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166b117-47dc-4934-9356-d50620c06828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:38.089408 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166b117-47dc-4934-9356-d50620c06828 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:38.089692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166b117-47dc-4934-9356-d50620c06828 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:38.090017 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166b117-47dc-4934-9356-d50620c06828 None admin] PATH_INFO: `/v3/groups/14581f2851d6447b8f3c0df3db37ca66` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:38.091415 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166b117-47dc-4934-9356-d50620c06828 None admin] RBAC: Authorizing `identity:get_group(group_id=14581f2851d6447b8f3c0df3db37ca66)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:38.092905 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166b117-47dc-4934-9356-d50620c06828 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:38.099365 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 191/376] 10.222.0.81 () {56 vars in 1180 bytes} [Tue Jun 7 14:07:37 2022] GET /identity/v3/groups/14581f2851d6447b8f3c0df3db37ca66 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:38.100942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1b122-d3e3-4f16-9cb0-f393f5f43826 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:38.101238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1b122-d3e3-4f16-9cb0-f393f5f43826 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:38.101595 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1b122-d3e3-4f16-9cb0-f393f5f43826 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:38.103854 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 192/377] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:38.105431 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1b122-d3e3-4f16-9cb0-f393f5f43826 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:38.192846 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13185846-b147-406d-9f19-094c82fd68c3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:38.237762 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0698d1c6-ea9d-4d4c-a728-1002cddc6099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:38.238639 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698d1c6-ea9d-4d4c-a728-1002cddc6099 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:38.238915 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698d1c6-ea9d-4d4c-a728-1002cddc6099 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:38.239183 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698d1c6-ea9d-4d4c-a728-1002cddc6099 None admin] PATH_INFO: `/v3/projects/50e31e6b5bc94b188158e931e4ee4076` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:38.240147 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1b122-d3e3-4f16-9cb0-f393f5f43826 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:38.240506 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0698d1c6-ea9d-4d4c-a728-1002cddc6099 None admin] RBAC: Authorizing `identity:get_project(project_id=50e31e6b5bc94b188158e931e4ee4076)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:38.242170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0698d1c6-ea9d-4d4c-a728-1002cddc6099 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:38.244830 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 186/378] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:38 2022] GET /identity/v3/projects/50e31e6b5bc94b188158e931e4ee4076 => generated 344 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:38.248441 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34e4fb-8764-4ae3-89be-98e4754495d3 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:38.248722 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34e4fb-8764-4ae3-89be-98e4754495d3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:38.248996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34e4fb-8764-4ae3-89be-98e4754495d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:38.269472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-fb34e4fb-8764-4ae3-89be-98e4754495d3 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:38.270663 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fb34e4fb-8764-4ae3-89be-98e4754495d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:38.289724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7dd88e-46d1-4bb5-bbc6-29cbed67b20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:38.290633 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7dd88e-46d1-4bb5-bbc6-29cbed67b20a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:38.290930 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7dd88e-46d1-4bb5-bbc6-29cbed67b20a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:38.291325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7dd88e-46d1-4bb5-bbc6-29cbed67b20a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:38.292187 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7dd88e-46d1-4bb5-bbc6-29cbed67b20a None admin] RBAC: Adding query filter params (group.id=14581f2851d6447b8f3c0df3db37ca66, role.id=6d02db1fe03b447c9e519f4f2aa16574, scope.project.id=c05a3987827c46a69dd1ab7b7f4b913b) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:38.292505 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7dd88e-46d1-4bb5-bbc6-29cbed67b20a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:38.294070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7dd88e-46d1-4bb5-bbc6-29cbed67b20a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:38.300361 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 193/379] 10.222.0.81 () {56 vars in 1376 bytes} [Tue Jun 7 14:07:38 2022] GET /identity/v3/role_assignments?group.id=14581f2851d6447b8f3c0df3db37ca66&scope.project.id=c05a3987827c46a69dd1ab7b7f4b913b&role.id=6d02db1fe03b447c9e519f4f2aa16574 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:38.312345 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7dd88e-46d1-4bb5-bbc6-29cbed67b20a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:38.343591 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:38.347782 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:38.354005 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 187/380] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:38.377602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb34e4fb-8764-4ae3-89be-98e4754495d3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:38.522114 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb34e4fb-8764-4ae3-89be-98e4754495d3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:38.523742 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7dd88e-46d1-4bb5-bbc6-29cbed67b20a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:38.563395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed65375-f686-4890-a10a-8c34fb207ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:38.564542 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed65375-f686-4890-a10a-8c34fb207ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:38.565054 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed65375-f686-4890-a10a-8c34fb207ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:38.565462 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed65375-f686-4890-a10a-8c34fb207ac4 None admin] PATH_INFO: `/v3/groups/ee975d59db9348eab48f086cff604a7e` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:38.577056 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed65375-f686-4890-a10a-8c34fb207ac4 None admin] RBAC: Authorizing `identity:get_group(group_id=ee975d59db9348eab48f086cff604a7e)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:38.579678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed65375-f686-4890-a10a-8c34fb207ac4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:38.581800 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 194/381] 10.222.0.81 () {56 vars in 1180 bytes} [Tue Jun 7 14:07:38 2022] GET /identity/v3/groups/ee975d59db9348eab48f086cff604a7e => generated 229 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:38.585922 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b5a07-881c-4453-bf22-26e9ff31b06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:38.587087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b5a07-881c-4453-bf22-26e9ff31b06f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:38.588174 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed65375-f686-4890-a10a-8c34fb207ac4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:38.588993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b5a07-881c-4453-bf22-26e9ff31b06f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:38.589777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b5a07-881c-4453-bf22-26e9ff31b06f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:38.616416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439b5a07-881c-4453-bf22-26e9ff31b06f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:38.637170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-439b5a07-881c-4453-bf22-26e9ff31b06f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:38.637170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:38.637170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:38.637170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:38.637170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:38.637170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:38.637170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:38.637170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:38.637170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:38.637170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:38.637170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:38.637170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:38.637170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:38.637170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:38.637170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:38.637170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:38.638772 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:38.638772 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:38.638772 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:38.638772 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:38.638772 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:38.638772 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:38.638772 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:38.638772 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:38.638772 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:38.638772 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:38.638772 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:38.638772 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:38.638772 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:38.638772 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:38.638772 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:38.638772 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:38.639818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:38.639818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:38.639818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:38.639818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:38.639818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:38.639818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:38.639818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:38.639818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:38.639818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:38.639818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:38.639818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:38.639818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:38.644146 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 188/382] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:07:38.648219 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b5a07-881c-4453-bf22-26e9ff31b06f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:38.714126 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:38.741572 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed65375-f686-4890-a10a-8c34fb207ac4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:38.755623 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:38.794636 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002a3cb-c93c-4798-a991-5408443cdbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:38.795249 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002a3cb-c93c-4798-a991-5408443cdbc4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:38.795545 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002a3cb-c93c-4798-a991-5408443cdbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:38.795825 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002a3cb-c93c-4798-a991-5408443cdbc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:38.796678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c002a3cb-c93c-4798-a991-5408443cdbc4 None admin] RBAC: Adding query filter params (group.id=ee975d59db9348eab48f086cff604a7e, role.id=82bbe3651b9b4f378fd150d93357b2b1, scope.project.id=50e31e6b5bc94b188158e931e4ee4076) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:38.796988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c002a3cb-c93c-4798-a991-5408443cdbc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:38.798429 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c002a3cb-c93c-4798-a991-5408443cdbc4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:38.809624 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 195/383] 10.222.0.81 () {56 vars in 1376 bytes} [Tue Jun 7 14:07:38 2022] GET /identity/v3/role_assignments?group.id=ee975d59db9348eab48f086cff604a7e&scope.project.id=50e31e6b5bc94b188158e931e4ee4076&role.id=82bbe3651b9b4f378fd150d93357b2b1 => generated 260 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:38.828927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b5a07-881c-4453-bf22-26e9ff31b06f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:38.881113 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a6082-efa2-483e-801f-a8cc641d876a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:38.881816 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a6082-efa2-483e-801f-a8cc641d876a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:38.882101 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a6082-efa2-483e-801f-a8cc641d876a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:38.882376 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a6082-efa2-483e-801f-a8cc641d876a None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:38.883989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a6082-efa2-483e-801f-a8cc641d876a None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:38.884416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a6082-efa2-483e-801f-a8cc641d876a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:38.885803 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a6082-efa2-483e-801f-a8cc641d876a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:38.902151 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 189/384] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:07:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:38.907712 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002a3cb-c93c-4798-a991-5408443cdbc4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:39.062003 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002a3cb-c93c-4798-a991-5408443cdbc4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:39.107790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4cced-8e49-4822-b59c-eb67fa318e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:39.108668 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4cced-8e49-4822-b59c-eb67fa318e26 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:39.108950 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4cced-8e49-4822-b59c-eb67fa318e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:39.109224 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4cced-8e49-4822-b59c-eb67fa318e26 None admin] PATH_INFO: `/v3/groups/14581f2851d6447b8f3c0df3db37ca66` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:39.110415 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d4cced-8e49-4822-b59c-eb67fa318e26 None admin] RBAC: Authorizing `identity:get_group(group_id=14581f2851d6447b8f3c0df3db37ca66)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:39.111892 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d4cced-8e49-4822-b59c-eb67fa318e26 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:39.113833 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 196/385] 10.222.0.81 () {56 vars in 1180 bytes} [Tue Jun 7 14:07:38 2022] GET /identity/v3/groups/14581f2851d6447b8f3c0df3db37ca66 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:39.119330 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a6082-efa2-483e-801f-a8cc641d876a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:39.234559 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a6082-efa2-483e-801f-a8cc641d876a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:39.259887 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:39.271863 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37080d-f972-44b4-b4ae-02df2cadd0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:39.273092 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37080d-f972-44b4-b4ae-02df2cadd0cd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:39.274544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37080d-f972-44b4-b4ae-02df2cadd0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:39.274944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37080d-f972-44b4-b4ae-02df2cadd0cd None admin] PATH_INFO: `/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:39.276188 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a37080d-f972-44b4-b4ae-02df2cadd0cd None admin] RBAC: Authorizing `identity:get_project(project_id=a85ae30482264d7ba1c84ab0ca46d4e0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:39.277654 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a37080d-f972-44b4-b4ae-02df2cadd0cd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:39.279713 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 190/386] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:39 2022] GET /identity/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:39.285031 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4cced-8e49-4822-b59c-eb67fa318e26 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:39.305007 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:39.450365 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4cced-8e49-4822-b59c-eb67fa318e26 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:39.503904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34495a-52fb-42a9-a9d5-75122afb92d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:39.504806 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34495a-52fb-42a9-a9d5-75122afb92d9 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:39.505085 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34495a-52fb-42a9-a9d5-75122afb92d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:39.505356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34495a-52fb-42a9-a9d5-75122afb92d9 None admin] PATH_INFO: `/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0/groups/14581f2851d6447b8f3c0df3db37ca66/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:39.507335 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b34495a-52fb-42a9-a9d5-75122afb92d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=a85ae30482264d7ba1c84ab0ca46d4e0, group_id=14581f2851d6447b8f3c0df3db37ca66, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:39.509810 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b34495a-52fb-42a9-a9d5-75122afb92d9 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:39.531978 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 197/387] 10.222.0.81 () {58 vars in 1442 bytes} [Tue Jun 7 14:07:39 2022] PUT /identity/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0/groups/14581f2851d6447b8f3c0df3db37ca66/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:07:40.138057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe326ba-183b-4dfd-9f9c-f078df0d6ac4 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:40.138569 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe326ba-183b-4dfd-9f9c-f078df0d6ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:40.138647 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe326ba-183b-4dfd-9f9c-f078df0d6ac4 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:40.140286 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 191/388] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:40.145177 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6308443-0676-4913-8fdb-be817aae7ef4 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:40.145572 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6308443-0676-4913-8fdb-be817aae7ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:40.145758 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6308443-0676-4913-8fdb-be817aae7ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:40.172221 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-e6308443-0676-4913-8fdb-be817aae7ef4 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:40.173437 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e6308443-0676-4913-8fdb-be817aae7ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:40.278588 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:40.287355 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 198/389] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:40.320983 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe326ba-183b-4dfd-9f9c-f078df0d6ac4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:40.453039 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe326ba-183b-4dfd-9f9c-f078df0d6ac4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:40.498583 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b40fd9c-0318-4ecb-98e3-48453b590567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:40.499417 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40fd9c-0318-4ecb-98e3-48453b590567 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:40.499696 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40fd9c-0318-4ecb-98e3-48453b590567 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:40.499966 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40fd9c-0318-4ecb-98e3-48453b590567 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:40.516302 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b40fd9c-0318-4ecb-98e3-48453b590567 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:40.524662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-9b40fd9c-0318-4ecb-98e3-48453b590567 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:07:40.524662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:40.524662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:40.524662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:40.524662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:40.524662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:40.524662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:40.524662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:40.524662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:40.524662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:40.524662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:40.524662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:40.524662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:40.524662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:40.524662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:40.524662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:40.525560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:40.525560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:40.525560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:40.525560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:40.525560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:40.525560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:40.525560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:40.525560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:40.525560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:40.525560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:40.525560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:40.525560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:40.525560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:40.525560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:40.525560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:40.525560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:40.526845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:40.526845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:40.526845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:40.526845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:40.526845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:40.526845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:40.526845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:40.526845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:40.526845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:40.526845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:40.526845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:07:40.526845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:40.526845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 192/390] 10.222.0.81 () {56 vars in 1114 bytes} [Tue Jun 7 14:07:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:40.533695 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6308443-0676-4913-8fdb-be817aae7ef4 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:40.698825 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfc723-d92c-40b2-ab07-f4006dae6d86 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:40.699188 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfc723-d92c-40b2-ab07-f4006dae6d86 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:40.699419 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfc723-d92c-40b2-ab07-f4006dae6d86 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:40.701006 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 193/391] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:40.703163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6308443-0676-4913-8fdb-be817aae7ef4 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:40.714106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56a1e4-4ada-463f-b06c-28a4f1bcd1e7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:40.714399 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56a1e4-4ada-463f-b06c-28a4f1bcd1e7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:40.714662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56a1e4-4ada-463f-b06c-28a4f1bcd1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:40.741664 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-fb56a1e4-4ada-463f-b06c-28a4f1bcd1e7 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:40.743623 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fb56a1e4-4ada-463f-b06c-28a4f1bcd1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:40.773837 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02fe4b-e9e4-4109-9e2a-336dc4ab8b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:40.774521 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02fe4b-e9e4-4109-9e2a-336dc4ab8b1a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:40.774804 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02fe4b-e9e4-4109-9e2a-336dc4ab8b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:40.775078 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02fe4b-e9e4-4109-9e2a-336dc4ab8b1a None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:40.775863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de02fe4b-e9e4-4109-9e2a-336dc4ab8b1a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:40.776167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de02fe4b-e9e4-4109-9e2a-336dc4ab8b1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:40.777548 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de02fe4b-e9e4-4109-9e2a-336dc4ab8b1a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:40.791578 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 199/392] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:07:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:40.797973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02fe4b-e9e4-4109-9e2a-336dc4ab8b1a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:40.846040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:40.891162 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 194/393] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:07:40.907485 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:40.919139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56a1e4-4ada-463f-b06c-28a4f1bcd1e7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:41.006042 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02fe4b-e9e4-4109-9e2a-336dc4ab8b1a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:41.048942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56a1e4-4ada-463f-b06c-28a4f1bcd1e7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:41.086450 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff165a-49ea-480b-a39a-4acd28ce0784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:41.087423 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff165a-49ea-480b-a39a-4acd28ce0784 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:41.087760 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff165a-49ea-480b-a39a-4acd28ce0784 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:41.088186 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff165a-49ea-480b-a39a-4acd28ce0784 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:41.099131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff165a-49ea-480b-a39a-4acd28ce0784 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:41.108955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-55ff165a-49ea-480b-a39a-4acd28ce0784 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:41.108955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:41.108955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:41.108955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:41.108955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:41.108955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:41.108955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:41.108955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:41.108955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:41.108955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:41.108955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:41.108955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:41.108955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:41.108955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:41.108955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:41.108955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:41.109903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:41.109903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:41.109903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:41.109903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:41.109903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:41.109903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:41.109903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:41.109903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:41.109903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:41.109903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:41.109903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:41.109903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:41.109903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:41.109903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:41.109903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:41.109903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:41.110868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:41.110868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:41.110868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:41.110868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:41.110868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:41.110868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:41.110868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:41.110868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:41.110868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:41.110868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:41.110868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:41.110868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:41.110868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 195/394] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:41.117186 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff165a-49ea-480b-a39a-4acd28ce0784 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:41.129234 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457926c2-800f-4c6e-8ff9-b7c0039cf31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:41.130102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457926c2-800f-4c6e-8ff9-b7c0039cf31c None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:41.130381 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457926c2-800f-4c6e-8ff9-b7c0039cf31c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:41.130649 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457926c2-800f-4c6e-8ff9-b7c0039cf31c None admin] PATH_INFO: `/v3/groups/14581f2851d6447b8f3c0df3db37ca66` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:41.132475 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457926c2-800f-4c6e-8ff9-b7c0039cf31c None admin] RBAC: Authorizing `identity:get_group(group_id=14581f2851d6447b8f3c0df3db37ca66)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:41.133946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457926c2-800f-4c6e-8ff9-b7c0039cf31c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:41.137822 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 200/395] 10.222.0.81 () {56 vars in 1180 bytes} [Tue Jun 7 14:07:40 2022] GET /identity/v3/groups/14581f2851d6447b8f3c0df3db37ca66 => generated 226 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:41.142081 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457926c2-800f-4c6e-8ff9-b7c0039cf31c None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:41.199647 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:41.308088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff165a-49ea-480b-a39a-4acd28ce0784 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:41.322179 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457926c2-800f-4c6e-8ff9-b7c0039cf31c None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:41.343521 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb410497-ef23-4f53-a96d-37439963a995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:41.344194 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb410497-ef23-4f53-a96d-37439963a995 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:41.344479 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb410497-ef23-4f53-a96d-37439963a995 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:41.344752 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb410497-ef23-4f53-a96d-37439963a995 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:41.345947 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb410497-ef23-4f53-a96d-37439963a995 None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:41.346259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb410497-ef23-4f53-a96d-37439963a995 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:41.347935 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb410497-ef23-4f53-a96d-37439963a995 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:41.359236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 196/396] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:07:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:41.359579 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:41.363210 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba47f6b-48b4-4637-9a78-999f8bdc7678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:41.364613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba47f6b-48b4-4637-9a78-999f8bdc7678 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:41.364969 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba47f6b-48b4-4637-9a78-999f8bdc7678 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:41.365338 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba47f6b-48b4-4637-9a78-999f8bdc7678 None admin] PATH_INFO: `/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:41.365513 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d9bf0-90ab-4014-a8de-c1d8e2e1b3a0 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:41.365927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d9bf0-90ab-4014-a8de-c1d8e2e1b3a0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:41.366215 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d9bf0-90ab-4014-a8de-c1d8e2e1b3a0 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:41.366874 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba47f6b-48b4-4637-9a78-999f8bdc7678 None admin] RBAC: Authorizing `identity:get_project(project_id=a85ae30482264d7ba1c84ab0ca46d4e0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:41.368426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 197/397] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:41.369132 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516d9bf0-90ab-4014-a8de-c1d8e2e1b3a0 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:41.373241 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba47f6b-48b4-4637-9a78-999f8bdc7678 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:41.376707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 201/398] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:41 2022] GET /identity/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0 => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:41.378024 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0cee66-a932-4a58-acc4-9b43a8b8fe08 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:41.378309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0cee66-a932-4a58-acc4-9b43a8b8fe08 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:41.378589 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0cee66-a932-4a58-acc4-9b43a8b8fe08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:41.399625 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-9d0cee66-a932-4a58-acc4-9b43a8b8fe08 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:41.400836 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9d0cee66-a932-4a58-acc4-9b43a8b8fe08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:41.490702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 202/399] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:41.491649 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0cee66-a932-4a58-acc4-9b43a8b8fe08 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:41.501770 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516d9bf0-90ab-4014-a8de-c1d8e2e1b3a0 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:41.503304 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:41.540353 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de4eb43-ec3e-48d3-9d91-449bb2fa367d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:41.541546 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4eb43-ec3e-48d3-9d91-449bb2fa367d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:41.541828 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4eb43-ec3e-48d3-9d91-449bb2fa367d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:41.542108 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4eb43-ec3e-48d3-9d91-449bb2fa367d None admin] PATH_INFO: `/v3/groups/14581f2851d6447b8f3c0df3db37ca66` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:41.543344 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de4eb43-ec3e-48d3-9d91-449bb2fa367d None admin] RBAC: Authorizing `identity:get_group(group_id=14581f2851d6447b8f3c0df3db37ca66)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:41.544825 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de4eb43-ec3e-48d3-9d91-449bb2fa367d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:41.546571 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 198/400] 10.222.0.81 () {56 vars in 1180 bytes} [Tue Jun 7 14:07:41 2022] GET /identity/v3/groups/14581f2851d6447b8f3c0df3db37ca66 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:41.547652 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080a7f4-260f-4159-a90a-9b6f18469492 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:41.547921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080a7f4-260f-4159-a90a-9b6f18469492 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:41.548201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080a7f4-260f-4159-a90a-9b6f18469492 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:41.553143 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 199/401] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:41.553143 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080a7f4-260f-4159-a90a-9b6f18469492 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:41.607763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0cee66-a932-4a58-acc4-9b43a8b8fe08 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:41.634749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:41.644151 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f6a02-6a6f-4d69-8ce8-1b31f5d86a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:41.645034 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f6a02-6a6f-4d69-8ce8-1b31f5d86a83 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:41.645619 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f6a02-6a6f-4d69-8ce8-1b31f5d86a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:41.645619 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f6a02-6a6f-4d69-8ce8-1b31f5d86a83 None admin] PATH_INFO: `/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0/groups/14581f2851d6447b8f3c0df3db37ca66/roles/6d02db1fe03b447c9e519f4f2aa16574` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:41.647858 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f6a02-6a6f-4d69-8ce8-1b31f5d86a83 None admin] RBAC: Authorizing `identity:create_grant(project_id=a85ae30482264d7ba1c84ab0ca46d4e0, group_id=14581f2851d6447b8f3c0df3db37ca66, role_id=6d02db1fe03b447c9e519f4f2aa16574)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:41.649559 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f6a02-6a6f-4d69-8ce8-1b31f5d86a83 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:41.658507 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080a7f4-260f-4159-a90a-9b6f18469492 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:41.662458 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 203/402] 10.222.0.81 () {58 vars in 1442 bytes} [Tue Jun 7 14:07:41 2022] PUT /identity/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0/groups/14581f2851d6447b8f3c0df3db37ca66/roles/6d02db1fe03b447c9e519f4f2aa16574 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:07:41.676225 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f6a02-6a6f-4d69-8ce8-1b31f5d86a83 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:41.698855 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8f368-c20d-43ca-a578-f86d871e9013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:41.699750 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8f368-c20d-43ca-a578-f86d871e9013 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:41.700077 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8f368-c20d-43ca-a578-f86d871e9013 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:41.700355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8f368-c20d-43ca-a578-f86d871e9013 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:41.714406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df8f368-c20d-43ca-a578-f86d871e9013 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:41.725645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-0df8f368-c20d-43ca-a578-f86d871e9013 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:07:41.725645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:41.725645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:41.725645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:41.725645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:41.725645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:41.725645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:41.725645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:41.725645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:41.725645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:41.725645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:41.725645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:41.725645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:41.725645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:41.725645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:41.725645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:41.727375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:41.727375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:41.727375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:41.727375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:41.727375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:41.727375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:41.727375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:41.727375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:41.727375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:41.727375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:41.727375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:41.727375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:41.727375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:41.727375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:41.727375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:41.727375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:41.728495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:41.728495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:41.728495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:41.728495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:41.728495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:41.728495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:41.728495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:41.728495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:41.728495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:41.728495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:41.728495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:07:41.728495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:41.728495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 200/403] 10.222.0.81 () {56 vars in 1114 bytes} [Tue Jun 7 14:07:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:41.729197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee524d4e-339e-41b0-9f61-e3e94f3a8cc8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:41.729838 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee524d4e-339e-41b0-9f61-e3e94f3a8cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:41.729838 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee524d4e-339e-41b0-9f61-e3e94f3a8cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:41.752984 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ee524d4e-339e-41b0-9f61-e3e94f3a8cc8 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:41.754405 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ee524d4e-339e-41b0-9f61-e3e94f3a8cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:41.788535 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:41.821427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f6a02-6a6f-4d69-8ce8-1b31f5d86a83 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:41.845941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 201/404] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:41.847200 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee524d4e-339e-41b0-9f61-e3e94f3a8cc8 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:41.873697 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790f559-e670-468c-a974-4f44d8159311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:41.874511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790f559-e670-468c-a974-4f44d8159311 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:41.874781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790f559-e670-468c-a974-4f44d8159311 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:41.875059 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790f559-e670-468c-a974-4f44d8159311 None admin] PATH_INFO: `/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:41.876356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790f559-e670-468c-a974-4f44d8159311 None admin] RBAC: Authorizing `identity:get_project(project_id=c05a3987827c46a69dd1ab7b7f4b913b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:41.878457 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790f559-e670-468c-a974-4f44d8159311 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:41.880327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 204/405] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:41 2022] GET /identity/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:41.881186 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790f559-e670-468c-a974-4f44d8159311 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:41.956018 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:41.966605 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee524d4e-339e-41b0-9f61-e3e94f3a8cc8 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:42.003926 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2cf5f-25c2-445d-b53b-1926154b52d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:42.005115 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2cf5f-25c2-445d-b53b-1926154b52d0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:42.005621 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2cf5f-25c2-445d-b53b-1926154b52d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:42.006211 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2cf5f-25c2-445d-b53b-1926154b52d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:42.007153 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d2cf5f-25c2-445d-b53b-1926154b52d0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:42.007953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d2cf5f-25c2-445d-b53b-1926154b52d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:42.009722 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d2cf5f-25c2-445d-b53b-1926154b52d0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:42.017079 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790f559-e670-468c-a974-4f44d8159311 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:42.019097 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 202/406] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:07:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:42.020173 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2cf5f-25c2-445d-b53b-1926154b52d0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:42.057081 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:42.059625 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b0f58-f1e5-42b3-8a67-551c2d1b1dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:42.060699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b0f58-f1e5-42b3-8a67-551c2d1b1dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:42.061183 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b0f58-f1e5-42b3-8a67-551c2d1b1dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:42.061687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b0f58-f1e5-42b3-8a67-551c2d1b1dc4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:42.074462 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5b0f58-f1e5-42b3-8a67-551c2d1b1dc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:42.082666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-ec5b0f58-f1e5-42b3-8a67-551c2d1b1dc4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:07:42.082666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:42.082666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:42.082666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:42.082666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:42.082666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:42.082666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:42.082666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:42.082666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:42.082666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:42.082666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:42.082666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:42.082666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:42.082666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:42.082666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:42.082666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:42.083691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:42.083691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:42.083691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:42.083691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:42.083691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:42.083691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:42.083691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:42.083691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:42.083691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:42.083691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:42.083691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:42.083691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:42.083691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:42.083691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:42.083691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:42.083691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:42.084526 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:42.084526 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:42.084526 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:42.084526 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:42.084526 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:42.084526 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:42.084526 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:42.084526 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:42.084526 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:42.084526 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:42.084526 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:07:42.084526 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:42.087895 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 205/407] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:07:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:42.088928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b0f58-f1e5-42b3-8a67-551c2d1b1dc4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:42.163369 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2cf5f-25c2-445d-b53b-1926154b52d0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:42.210064 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b0f58-f1e5-42b3-8a67-551c2d1b1dc4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:42.213721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff182f-31df-4b1e-b9a6-79bf72d6381d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:42.215952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff182f-31df-4b1e-b9a6-79bf72d6381d None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:42.216386 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff182f-31df-4b1e-b9a6-79bf72d6381d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:42.216804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff182f-31df-4b1e-b9a6-79bf72d6381d None admin] PATH_INFO: `/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b/groups/14581f2851d6447b8f3c0df3db37ca66/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:42.219018 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eff182f-31df-4b1e-b9a6-79bf72d6381d None admin] RBAC: Authorizing `identity:create_grant(project_id=c05a3987827c46a69dd1ab7b7f4b913b, group_id=14581f2851d6447b8f3c0df3db37ca66, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:42.221063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eff182f-31df-4b1e-b9a6-79bf72d6381d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:42.228631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:42.239852 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 203/408] 10.222.0.81 () {58 vars in 1442 bytes} [Tue Jun 7 14:07:42 2022] PUT /identity/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b/groups/14581f2851d6447b8f3c0df3db37ca66/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:07:42.240873 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff182f-31df-4b1e-b9a6-79bf72d6381d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:42.279489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041ee5c-0b7e-4a63-8b1f-722a3edb2b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:42.280563 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041ee5c-0b7e-4a63-8b1f-722a3edb2b18 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:42.281320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041ee5c-0b7e-4a63-8b1f-722a3edb2b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:42.281320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041ee5c-0b7e-4a63-8b1f-722a3edb2b18 None admin] PATH_INFO: `/v3/groups/14581f2851d6447b8f3c0df3db37ca66` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:42.284029 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041ee5c-0b7e-4a63-8b1f-722a3edb2b18 None admin] RBAC: Authorizing `identity:get_group(group_id=14581f2851d6447b8f3c0df3db37ca66)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:42.285657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041ee5c-0b7e-4a63-8b1f-722a3edb2b18 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:42.287663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 206/409] 10.222.0.81 () {56 vars in 1180 bytes} [Tue Jun 7 14:07:42 2022] GET /identity/v3/groups/14581f2851d6447b8f3c0df3db37ca66 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:42.294010 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041ee5c-0b7e-4a63-8b1f-722a3edb2b18 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:42.379722 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:42.401001 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff182f-31df-4b1e-b9a6-79bf72d6381d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:42.419621 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041ee5c-0b7e-4a63-8b1f-722a3edb2b18 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:42.439324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699beb32-dc48-48c9-98cb-00be5f2aad35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:42.440009 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699beb32-dc48-48c9-98cb-00be5f2aad35 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:42.440299 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699beb32-dc48-48c9-98cb-00be5f2aad35 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:42.440578 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699beb32-dc48-48c9-98cb-00be5f2aad35 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:42.441338 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699beb32-dc48-48c9-98cb-00be5f2aad35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:42.441653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699beb32-dc48-48c9-98cb-00be5f2aad35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:42.443145 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699beb32-dc48-48c9-98cb-00be5f2aad35 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:42.452618 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 204/410] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:42.459989 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d9eb2-56ff-4dd8-b4ac-0edca5e93307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:42.461406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699beb32-dc48-48c9-98cb-00be5f2aad35 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:42.461533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d9eb2-56ff-4dd8-b4ac-0edca5e93307 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:42.462116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d9eb2-56ff-4dd8-b4ac-0edca5e93307 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:42.462606 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d9eb2-56ff-4dd8-b4ac-0edca5e93307 None admin] PATH_INFO: `/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:42.464118 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d9eb2-56ff-4dd8-b4ac-0edca5e93307 None admin] RBAC: Authorizing `identity:get_project(project_id=c05a3987827c46a69dd1ab7b7f4b913b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:42.465901 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d9eb2-56ff-4dd8-b4ac-0edca5e93307 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:42.468167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 207/411] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:42 2022] GET /identity/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:42.473298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d9eb2-56ff-4dd8-b4ac-0edca5e93307 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:42.531843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:42.595245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d9eb2-56ff-4dd8-b4ac-0edca5e93307 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:42.625806 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699beb32-dc48-48c9-98cb-00be5f2aad35 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:42.633052 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc967c2-841f-473b-bb90-c385556f5d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:42.634121 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc967c2-841f-473b-bb90-c385556f5d51 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:42.634406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc967c2-841f-473b-bb90-c385556f5d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:42.634688 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc967c2-841f-473b-bb90-c385556f5d51 None admin] PATH_INFO: `/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b/groups/14581f2851d6447b8f3c0df3db37ca66/roles/6d02db1fe03b447c9e519f4f2aa16574` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:42.637099 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc967c2-841f-473b-bb90-c385556f5d51 None admin] RBAC: Authorizing `identity:create_grant(project_id=c05a3987827c46a69dd1ab7b7f4b913b, group_id=14581f2851d6447b8f3c0df3db37ca66, role_id=6d02db1fe03b447c9e519f4f2aa16574)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:42.638646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc967c2-841f-473b-bb90-c385556f5d51 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:42.655655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 208/412] 10.222.0.81 () {58 vars in 1442 bytes} [Tue Jun 7 14:07:42 2022] PUT /identity/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b/groups/14581f2851d6447b8f3c0df3db37ca66/roles/6d02db1fe03b447c9e519f4f2aa16574 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:07:42.659578 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f8232-84cb-43a2-9185-3775f06e6364 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:42.659897 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f8232-84cb-43a2-9185-3775f06e6364 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:42.660230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f8232-84cb-43a2-9185-3775f06e6364 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:42.667944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 209/413] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:42.670023 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bcf9fd-7236-4c73-87b9-ca19b7702ae9 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:42.670254 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bcf9fd-7236-4c73-87b9-ca19b7702ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:42.670534 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bcf9fd-7236-4c73-87b9-ca19b7702ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:42.684888 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d056de-68a1-4e48-9721-a2b00b82202c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:42.687027 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d056de-68a1-4e48-9721-a2b00b82202c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:42.687391 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d056de-68a1-4e48-9721-a2b00b82202c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:42.689413 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d056de-68a1-4e48-9721-a2b00b82202c None admin] PATH_INFO: `/v3/groups/ee975d59db9348eab48f086cff604a7e` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:42.693061 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d056de-68a1-4e48-9721-a2b00b82202c None admin] RBAC: Authorizing `identity:get_group(group_id=ee975d59db9348eab48f086cff604a7e)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:42.695329 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d056de-68a1-4e48-9721-a2b00b82202c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:42.700065 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 205/414] 10.222.0.81 () {56 vars in 1180 bytes} [Tue Jun 7 14:07:42 2022] GET /identity/v3/groups/ee975d59db9348eab48f086cff604a7e => generated 229 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:42.707172 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d056de-68a1-4e48-9721-a2b00b82202c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:42.708546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-48bcf9fd-7236-4c73-87b9-ca19b7702ae9 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:42.709764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-48bcf9fd-7236-4c73-87b9-ca19b7702ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:42.737698 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:42.807410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 210/415] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:42.831115 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bcf9fd-7236-4c73-87b9-ca19b7702ae9 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:42.921120 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d056de-68a1-4e48-9721-a2b00b82202c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:42.923606 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:42.968998 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6d1e2-a34a-404a-9789-5b22d0eb5a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:42.969872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6d1e2-a34a-404a-9789-5b22d0eb5a83 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:42.970153 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6d1e2-a34a-404a-9789-5b22d0eb5a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:42.970424 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6d1e2-a34a-404a-9789-5b22d0eb5a83 None admin] PATH_INFO: `/v3/projects/50e31e6b5bc94b188158e931e4ee4076` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:42.971668 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c6d1e2-a34a-404a-9789-5b22d0eb5a83 None admin] RBAC: Authorizing `identity:get_project(project_id=50e31e6b5bc94b188158e931e4ee4076)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:42.973121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c6d1e2-a34a-404a-9789-5b22d0eb5a83 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:42.975649 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 206/416] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:42 2022] GET /identity/v3/projects/50e31e6b5bc94b188158e931e4ee4076 => generated 344 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:42.981781 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6d1e2-a34a-404a-9789-5b22d0eb5a83 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:43.005836 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bcf9fd-7236-4c73-87b9-ca19b7702ae9 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:43.056983 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e716ba-8ea2-468c-83ad-da51dcfbd659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:43.060480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e716ba-8ea2-468c-83ad-da51dcfbd659 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:43.061207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e716ba-8ea2-468c-83ad-da51dcfbd659 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:43.061689 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e716ba-8ea2-468c-83ad-da51dcfbd659 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:43.075753 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e716ba-8ea2-468c-83ad-da51dcfbd659 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:43.084923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-71e716ba-8ea2-468c-83ad-da51dcfbd659 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:43.084923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:43.084923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:43.084923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:43.084923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:43.084923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:43.084923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:43.084923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:43.084923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:43.084923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:43.084923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:43.084923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:43.084923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:43.084923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:43.084923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:43.084923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:43.085916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:43.085916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:43.085916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:43.085916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:43.085916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:43.085916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:43.085916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:43.085916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:43.085916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:43.085916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:43.085916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:43.085916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:43.085916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:43.085916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:43.085916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:43.085916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:43.086930 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:43.086930 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:43.086930 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:43.086930 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:43.086930 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:43.086930 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:43.086930 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:43.086930 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:43.086930 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:43.086930 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:43.086930 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:43.086930 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:43.094843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 211/417] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:43.104432 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e716ba-8ea2-468c-83ad-da51dcfbd659 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:43.153346 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:43.222225 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6d1e2-a34a-404a-9789-5b22d0eb5a83 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:43.227311 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:43.231243 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e716ba-8ea2-468c-83ad-da51dcfbd659 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:43.275244 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e597cd-abd3-4556-bed4-b405a15d50a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:43.275244 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e597cd-abd3-4556-bed4-b405a15d50a7 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:43.275244 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e597cd-abd3-4556-bed4-b405a15d50a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:43.275244 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e597cd-abd3-4556-bed4-b405a15d50a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:43.275644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e597cd-abd3-4556-bed4-b405a15d50a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:43.275644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e597cd-abd3-4556-bed4-b405a15d50a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:43.275961 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e597cd-abd3-4556-bed4-b405a15d50a7 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:43.300341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 212/418] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:07:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:43.305008 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43e3b8-8bc0-4bc3-a080-861d3f2c4bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:43.306182 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43e3b8-8bc0-4bc3-a080-861d3f2c4bde None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:43.307261 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e597cd-abd3-4556-bed4-b405a15d50a7 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:43.308513 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43e3b8-8bc0-4bc3-a080-861d3f2c4bde None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:43.308993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43e3b8-8bc0-4bc3-a080-861d3f2c4bde None admin] PATH_INFO: `/v3/projects/50e31e6b5bc94b188158e931e4ee4076/groups/ee975d59db9348eab48f086cff604a7e/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:43.312828 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43e3b8-8bc0-4bc3-a080-861d3f2c4bde None admin] RBAC: Authorizing `identity:create_grant(project_id=50e31e6b5bc94b188158e931e4ee4076, group_id=ee975d59db9348eab48f086cff604a7e, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:43.315548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43e3b8-8bc0-4bc3-a080-861d3f2c4bde None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:43.342179 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 207/419] 10.222.0.81 () {58 vars in 1442 bytes} [Tue Jun 7 14:07:42 2022] PUT /identity/v3/projects/50e31e6b5bc94b188158e931e4ee4076/groups/ee975d59db9348eab48f086cff604a7e/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:07:43.439757 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e597cd-abd3-4556-bed4-b405a15d50a7 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:43.483944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bdd6f9-b61d-457e-8af5-7cfb88e1568a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:43.484966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdd6f9-b61d-457e-8af5-7cfb88e1568a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:43.485452 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdd6f9-b61d-457e-8af5-7cfb88e1568a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:43.485929 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdd6f9-b61d-457e-8af5-7cfb88e1568a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:43.521203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:43.569243 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bdd6f9-b61d-457e-8af5-7cfb88e1568a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:43.572072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bdd6f9-b61d-457e-8af5-7cfb88e1568a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:43.674294 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 213/420] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:07:43.680441 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43e3b8-8bc0-4bc3-a080-861d3f2c4bde None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:43.884759 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43e3b8-8bc0-4bc3-a080-861d3f2c4bde None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:43.930672 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330c1058-ec03-4b74-8513-a98d6ef93656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:43.931921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c1058-ec03-4b74-8513-a98d6ef93656 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:43.932426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c1058-ec03-4b74-8513-a98d6ef93656 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:43.932891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c1058-ec03-4b74-8513-a98d6ef93656 None admin] PATH_INFO: `/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:43.934294 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330c1058-ec03-4b74-8513-a98d6ef93656 None admin] RBAC: Authorizing `identity:get_project(project_id=a85ae30482264d7ba1c84ab0ca46d4e0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:43.935954 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330c1058-ec03-4b74-8513-a98d6ef93656 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:43.938149 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 208/421] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:43 2022] GET /identity/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0 => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:43.943455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bdd6f9-b61d-457e-8af5-7cfb88e1568a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:44.055608 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:44.110872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bdd6f9-b61d-457e-8af5-7cfb88e1568a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:44.174282 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea653fe-67be-4f38-bb6d-0eadfc2fd737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:44.175151 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea653fe-67be-4f38-bb6d-0eadfc2fd737 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:44.175453 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea653fe-67be-4f38-bb6d-0eadfc2fd737 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:44.175725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea653fe-67be-4f38-bb6d-0eadfc2fd737 None admin] PATH_INFO: `/v3/groups/14581f2851d6447b8f3c0df3db37ca66` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:44.180652 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea653fe-67be-4f38-bb6d-0eadfc2fd737 None admin] RBAC: Authorizing `identity:get_group(group_id=14581f2851d6447b8f3c0df3db37ca66)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:44.182080 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea653fe-67be-4f38-bb6d-0eadfc2fd737 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:44.184075 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 214/422] 10.222.0.81 () {56 vars in 1180 bytes} [Tue Jun 7 14:07:43 2022] GET /identity/v3/groups/14581f2851d6447b8f3c0df3db37ca66 => generated 226 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:44.189662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330c1058-ec03-4b74-8513-a98d6ef93656 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:44.205671 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfeebe1-c6ec-4ca3-93bc-1120bdafa4df None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:44.205962 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfeebe1-c6ec-4ca3-93bc-1120bdafa4df None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:44.206231 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfeebe1-c6ec-4ca3-93bc-1120bdafa4df None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:44.208178 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 215/423] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:44.208322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:44.217160 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e9a86-b4d3-4707-9d86-41b541fa7fc9 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:44.217598 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e9a86-b4d3-4707-9d86-41b541fa7fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:44.217971 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e9a86-b4d3-4707-9d86-41b541fa7fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:44.237434 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6d0e9a86-b4d3-4707-9d86-41b541fa7fc9 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:44.238760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6d0e9a86-b4d3-4707-9d86-41b541fa7fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:44.353501 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330c1058-ec03-4b74-8513-a98d6ef93656 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:44.361894 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 216/424] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:07:44.407576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b1b02-4592-4a2c-aa77-6f16bb049045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:44.408308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b1b02-4592-4a2c-aa77-6f16bb049045 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:44.408638 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b1b02-4592-4a2c-aa77-6f16bb049045 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:44.408919 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b1b02-4592-4a2c-aa77-6f16bb049045 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:44.409763 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b1b02-4592-4a2c-aa77-6f16bb049045 None admin] RBAC: Adding query filter params (group.id=14581f2851d6447b8f3c0df3db37ca66, role.id=ca91f64e7a46431c8aa4285c4620f216, scope.project.id=a85ae30482264d7ba1c84ab0ca46d4e0) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:44.410074 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b1b02-4592-4a2c-aa77-6f16bb049045 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:44.411533 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b1b02-4592-4a2c-aa77-6f16bb049045 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:44.423791 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 209/425] 10.222.0.81 () {56 vars in 1376 bytes} [Tue Jun 7 14:07:44 2022] GET /identity/v3/role_assignments?group.id=14581f2851d6447b8f3c0df3db37ca66&scope.project.id=a85ae30482264d7ba1c84ab0ca46d4e0&role.id=ca91f64e7a46431c8aa4285c4620f216 => generated 611 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:44.426422 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e9a86-b4d3-4707-9d86-41b541fa7fc9 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:44.514786 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:44.574691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e9a86-b4d3-4707-9d86-41b541fa7fc9 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:44.624550 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af7928-ceeb-40dc-8705-0352cebdbc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:44.625381 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af7928-ceeb-40dc-8705-0352cebdbc45 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:44.625659 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af7928-ceeb-40dc-8705-0352cebdbc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:44.625929 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af7928-ceeb-40dc-8705-0352cebdbc45 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:44.642203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13af7928-ceeb-40dc-8705-0352cebdbc45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:44.650690 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-13af7928-ceeb-40dc-8705-0352cebdbc45 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:07:44.650690 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:44.650690 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:44.650690 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:44.650690 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:44.650690 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:44.650690 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:44.650690 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:44.650690 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:44.650690 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:44.650690 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:44.650690 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:44.650690 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:44.650690 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:44.650690 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:44.650690 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:44.651610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:44.651610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:44.651610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:44.651610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:44.651610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:44.651610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:44.651610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:44.651610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:44.651610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:44.651610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:44.651610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:44.651610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:44.651610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:44.651610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:44.651610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:44.651610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:44.652410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:44.652410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:44.652410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:44.652410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:44.652410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:44.652410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:44.652410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:44.652410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:44.652410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:44.652410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:44.652410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:07:44.652410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:44.652410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 217/426] 10.222.0.81 () {56 vars in 1114 bytes} [Tue Jun 7 14:07:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:44.659179 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b1b02-4592-4a2c-aa77-6f16bb049045 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:44.734109 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:44.768147 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b1b02-4592-4a2c-aa77-6f16bb049045 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:44.806984 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5eca7a-5727-4d61-b5b0-37e71d94dde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:44.807762 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5eca7a-5727-4d61-b5b0-37e71d94dde2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:44.808088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5eca7a-5727-4d61-b5b0-37e71d94dde2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:44.808416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5eca7a-5727-4d61-b5b0-37e71d94dde2 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:44.809181 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5eca7a-5727-4d61-b5b0-37e71d94dde2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:44.809485 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5eca7a-5727-4d61-b5b0-37e71d94dde2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:44.810852 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5eca7a-5727-4d61-b5b0-37e71d94dde2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:44.819654 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 210/427] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:07:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:44.825544 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af7928-ceeb-40dc-8705-0352cebdbc45 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:44.938550 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af7928-ceeb-40dc-8705-0352cebdbc45 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:44.958814 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:44.977233 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188bee5-ae2d-49c2-a47b-dea3b3fe99b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:44.977996 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188bee5-ae2d-49c2-a47b-dea3b3fe99b2 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:44.978279 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188bee5-ae2d-49c2-a47b-dea3b3fe99b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:44.978558 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188bee5-ae2d-49c2-a47b-dea3b3fe99b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:45.056300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188bee5-ae2d-49c2-a47b-dea3b3fe99b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:45.057766 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188bee5-ae2d-49c2-a47b-dea3b3fe99b2 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:45.163269 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bcf93-847f-4742-bb77-5c38daab9b27 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:45.163269 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bcf93-847f-4742-bb77-5c38daab9b27 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:45.163269 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bcf93-847f-4742-bb77-5c38daab9b27 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:45.166143 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 211/428] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:07:45.173046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c3000-991c-4e06-aab3-ddc9efadd100 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:45.173647 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c3000-991c-4e06-aab3-ddc9efadd100 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:45.174117 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c3000-991c-4e06-aab3-ddc9efadd100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:45.185615 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 218/429] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:07:45.192297 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188bee5-ae2d-49c2-a47b-dea3b3fe99b2 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:45.203556 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-447c3000-991c-4e06-aab3-ddc9efadd100 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:45.205049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-447c3000-991c-4e06-aab3-ddc9efadd100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:45.227036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:45.289390 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 212/430] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:07:45.316451 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c3000-991c-4e06-aab3-ddc9efadd100 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:45.327298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188bee5-ae2d-49c2-a47b-dea3b3fe99b2 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:45.376491 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21a0ac-0c54-4580-8681-1562980fc6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:45.377701 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21a0ac-0c54-4580-8681-1562980fc6bc None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:45.377701 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21a0ac-0c54-4580-8681-1562980fc6bc None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:45.377969 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21a0ac-0c54-4580-8681-1562980fc6bc None admin] PATH_INFO: `/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:45.379388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21a0ac-0c54-4580-8681-1562980fc6bc None admin] RBAC: Authorizing `identity:get_project(project_id=a85ae30482264d7ba1c84ab0ca46d4e0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:45.381257 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21a0ac-0c54-4580-8681-1562980fc6bc None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:45.384615 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 219/431] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:45 2022] GET /identity/v3/projects/a85ae30482264d7ba1c84ab0ca46d4e0 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:45.393448 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21a0ac-0c54-4580-8681-1562980fc6bc None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:45.432322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:45.477721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c3000-991c-4e06-aab3-ddc9efadd100 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:45.514928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:45.518049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ac2ad-1129-4029-9240-283960fec325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:45.519065 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ac2ad-1129-4029-9240-283960fec325 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:45.519544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ac2ad-1129-4029-9240-283960fec325 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:45.519983 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ac2ad-1129-4029-9240-283960fec325 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:45.531184 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21a0ac-0c54-4580-8681-1562980fc6bc None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:45.534253 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ac2ad-1129-4029-9240-283960fec325 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:45.542971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-764ac2ad-1129-4029-9240-283960fec325 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:45.542971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:45.542971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:45.542971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:45.542971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:45.542971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:45.542971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:45.542971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:45.542971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:45.542971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:45.542971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:45.542971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:45.542971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:45.542971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:45.542971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:45.542971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:45.543942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:45.543942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:45.543942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:45.543942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:45.543942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:45.543942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:45.543942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:45.543942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:45.543942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:45.543942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:45.543942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:45.543942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:45.543942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:45.543942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:45.543942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:45.543942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:45.544723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:45.544723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:45.544723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:45.544723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:45.544723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:45.544723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:45.544723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:45.544723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:45.544723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:45.544723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:45.544723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:07:45.544723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:45.559944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 213/432] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:45.567020 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ac2ad-1129-4029-9240-283960fec325 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:45.581547 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10678232-507a-467e-85f4-0f445bf7d872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:45.582515 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10678232-507a-467e-85f4-0f445bf7d872 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:45.582888 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10678232-507a-467e-85f4-0f445bf7d872 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:45.583275 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10678232-507a-467e-85f4-0f445bf7d872 None admin] PATH_INFO: `/v3/groups/14581f2851d6447b8f3c0df3db37ca66` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:45.584511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10678232-507a-467e-85f4-0f445bf7d872 None admin] RBAC: Authorizing `identity:get_group(group_id=14581f2851d6447b8f3c0df3db37ca66)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:45.586743 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10678232-507a-467e-85f4-0f445bf7d872 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:45.594711 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 220/433] 10.222.0.81 () {56 vars in 1180 bytes} [Tue Jun 7 14:07:45 2022] GET /identity/v3/groups/14581f2851d6447b8f3c0df3db37ca66 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:45.596974 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10678232-507a-467e-85f4-0f445bf7d872 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:45.691436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ac2ad-1129-4029-9240-283960fec325 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:45.703944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10678232-507a-467e-85f4-0f445bf7d872 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:45.728319 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:45.743022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240acf1-d88d-43e2-94d4-50f97132c3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:45.743945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240acf1-d88d-43e2-94d4-50f97132c3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:45.744293 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49fa160-7d10-4d8a-bf98-dcf42295508a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:45.744635 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240acf1-d88d-43e2-94d4-50f97132c3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:45.745050 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240acf1-d88d-43e2-94d4-50f97132c3f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:45.745236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49fa160-7d10-4d8a-bf98-dcf42295508a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:45.745608 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49fa160-7d10-4d8a-bf98-dcf42295508a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:45.745976 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49fa160-7d10-4d8a-bf98-dcf42295508a None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:45.746739 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240acf1-d88d-43e2-94d4-50f97132c3f8 None admin] RBAC: Adding query filter params (group.id=14581f2851d6447b8f3c0df3db37ca66, role.id=6d02db1fe03b447c9e519f4f2aa16574, scope.project.id=a85ae30482264d7ba1c84ab0ca46d4e0) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:45.747185 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240acf1-d88d-43e2-94d4-50f97132c3f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:45.747376 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49fa160-7d10-4d8a-bf98-dcf42295508a None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:45.747775 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49fa160-7d10-4d8a-bf98-dcf42295508a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:45.749075 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240acf1-d88d-43e2-94d4-50f97132c3f8 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:45.749268 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49fa160-7d10-4d8a-bf98-dcf42295508a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:45.755886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 221/434] 10.222.0.81 () {56 vars in 1376 bytes} [Tue Jun 7 14:07:45 2022] GET /identity/v3/role_assignments?group.id=14581f2851d6447b8f3c0df3db37ca66&scope.project.id=a85ae30482264d7ba1c84ab0ca46d4e0&role.id=6d02db1fe03b447c9e519f4f2aa16574 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:45.757309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c01a2-7706-47c1-9108-97a990ad4ff5 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:45.757626 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c01a2-7706-47c1-9108-97a990ad4ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:45.757963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c01a2-7706-47c1-9108-97a990ad4ff5 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:45.759413 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 222/435] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:45.769887 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 214/436] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:07:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:45.770127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8bb6d-a9cf-4b8e-b8ea-19f3273202d6 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:45.770127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8bb6d-a9cf-4b8e-b8ea-19f3273202d6 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:45.770127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8bb6d-a9cf-4b8e-b8ea-19f3273202d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:45.775338 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49fa160-7d10-4d8a-bf98-dcf42295508a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:45.798689 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-cff8bb6d-a9cf-4b8e-b8ea-19f3273202d6 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:45.799923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-cff8bb6d-a9cf-4b8e-b8ea-19f3273202d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:45.829977 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:45.960362 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 223/437] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:45.978245 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49fa160-7d10-4d8a-bf98-dcf42295508a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:45.985394 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff8bb6d-a9cf-4b8e-b8ea-19f3273202d6 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:46.019943 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586b17a-0176-4167-8603-cce7ff66cd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:46.020600 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586b17a-0176-4167-8603-cce7ff66cd2a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:46.020887 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586b17a-0176-4167-8603-cce7ff66cd2a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:46.021164 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586b17a-0176-4167-8603-cce7ff66cd2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:46.102194 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:46.109729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff8bb6d-a9cf-4b8e-b8ea-19f3273202d6 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:46.118082 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c586b17a-0176-4167-8603-cce7ff66cd2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:46.120137 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c586b17a-0176-4167-8603-cce7ff66cd2a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:46.158493 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781603d-bca0-45c1-9478-5055c3271548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:46.159717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781603d-bca0-45c1-9478-5055c3271548 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:46.160200 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781603d-bca0-45c1-9478-5055c3271548 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:46.160662 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781603d-bca0-45c1-9478-5055c3271548 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:46.178060 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5781603d-bca0-45c1-9478-5055c3271548 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:46.189001 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-5781603d-bca0-45c1-9478-5055c3271548 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:07:46.189001 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:46.189001 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:46.189001 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:46.189001 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:46.189001 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:46.189001 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:46.189001 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:46.189001 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:46.189001 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:46.189001 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:46.189001 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:46.189001 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:46.189001 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:46.189001 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:46.189001 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:46.189953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:46.189953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:46.189953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:46.189953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:46.189953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:46.189953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:46.189953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:46.189953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:46.189953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:46.189953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:46.189953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:46.189953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:46.189953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:46.189953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:46.189953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:46.189953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:46.190784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:46.190784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:46.190784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:46.190784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:46.190784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:46.190784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:46.190784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:46.190784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:46.190784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:46.190784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:46.190784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:07:46.190784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:46.190784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 224/438] 10.222.0.81 () {56 vars in 1114 bytes} [Tue Jun 7 14:07:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:07:46.198343 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781603d-bca0-45c1-9478-5055c3271548 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:46.212963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:46.250591 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 215/439] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 14:07:46.252304 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56b3d2-dc4f-42ce-b807-203f63e17fa4 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:46.252597 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56b3d2-dc4f-42ce-b807-203f63e17fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:46.252869 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56b3d2-dc4f-42ce-b807-203f63e17fa4 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:46.254473 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 216/440] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:46.258529 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56b3d2-dc4f-42ce-b807-203f63e17fa4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:46.312027 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781603d-bca0-45c1-9478-5055c3271548 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:46.348747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb839b84-31a8-41df-951c-ec9b2a2b7c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:46.349414 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb839b84-31a8-41df-951c-ec9b2a2b7c06 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:46.349691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb839b84-31a8-41df-951c-ec9b2a2b7c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:46.349959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb839b84-31a8-41df-951c-ec9b2a2b7c06 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:46.350630 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:46.350739 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb839b84-31a8-41df-951c-ec9b2a2b7c06 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:46.350990 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb839b84-31a8-41df-951c-ec9b2a2b7c06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:46.353558 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb839b84-31a8-41df-951c-ec9b2a2b7c06 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:46.361603 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 225/441] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:07:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:46.362704 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832af89-8a4e-4629-8eda-b432f8fdef7f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:46.362985 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832af89-8a4e-4629-8eda-b432f8fdef7f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:46.363288 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832af89-8a4e-4629-8eda-b432f8fdef7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:46.370084 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56b3d2-dc4f-42ce-b807-203f63e17fa4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:46.379658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6832af89-8a4e-4629-8eda-b432f8fdef7f None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:46.380815 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6832af89-8a4e-4629-8eda-b432f8fdef7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:46.405095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5179938-3332-4ce3-bc56-9add71581a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:46.405948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5179938-3332-4ce3-bc56-9add71581a1c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:46.406238 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5179938-3332-4ce3-bc56-9add71581a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:46.406514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5179938-3332-4ce3-bc56-9add71581a1c None admin] PATH_INFO: `/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:46.407727 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5179938-3332-4ce3-bc56-9add71581a1c None admin] RBAC: Authorizing `identity:get_project(project_id=c05a3987827c46a69dd1ab7b7f4b913b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:46.409167 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5179938-3332-4ce3-bc56-9add71581a1c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:46.410900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 217/442] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:46 2022] GET /identity/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:46.411697 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5179938-3332-4ce3-bc56-9add71581a1c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:46.448122 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 226/443] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:46.448968 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832af89-8a4e-4629-8eda-b432f8fdef7f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:46.451362 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:46.518034 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5179938-3332-4ce3-bc56-9add71581a1c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:46.555554 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832af89-8a4e-4629-8eda-b432f8fdef7f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:46.556236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9f2815-742a-4fc0-8391-b9b82cc83624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:46.556866 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f2815-742a-4fc0-8391-b9b82cc83624 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:46.557155 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f2815-742a-4fc0-8391-b9b82cc83624 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:46.557433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f2815-742a-4fc0-8391-b9b82cc83624 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:46.581328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:46.591619 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803534b-588e-474f-be62-9ede6d7703b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:46.592474 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803534b-588e-474f-be62-9ede6d7703b4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:46.592759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803534b-588e-474f-be62-9ede6d7703b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:46.593030 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803534b-588e-474f-be62-9ede6d7703b4 None admin] PATH_INFO: `/v3/groups/14581f2851d6447b8f3c0df3db37ca66` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:46.594192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b803534b-588e-474f-be62-9ede6d7703b4 None admin] RBAC: Authorizing `identity:get_group(group_id=14581f2851d6447b8f3c0df3db37ca66)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:46.595661 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b803534b-588e-474f-be62-9ede6d7703b4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:46.597606 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 227/444] 10.222.0.81 () {56 vars in 1180 bytes} [Tue Jun 7 14:07:46 2022] GET /identity/v3/groups/14581f2851d6447b8f3c0df3db37ca66 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:46.598437 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803534b-588e-474f-be62-9ede6d7703b4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:46.626524 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9f2815-742a-4fc0-8391-b9b82cc83624 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:46.627997 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9f2815-742a-4fc0-8391-b9b82cc83624 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:46.669527 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:46.700461 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803534b-588e-474f-be62-9ede6d7703b4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:46.730769 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 218/445] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:07:46.730769 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9f2815-742a-4fc0-8391-b9b82cc83624 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:46.734101 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a02d36-085a-4eca-b700-cc33eba3e4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:46.734892 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a02d36-085a-4eca-b700-cc33eba3e4cc None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:46.735171 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a02d36-085a-4eca-b700-cc33eba3e4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:46.735484 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a02d36-085a-4eca-b700-cc33eba3e4cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:46.746369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a02d36-085a-4eca-b700-cc33eba3e4cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:46.753993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-67a02d36-085a-4eca-b700-cc33eba3e4cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:07:46.753993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:46.753993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:46.753993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:46.753993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:46.753993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:46.753993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:46.753993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:46.753993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:46.753993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:46.753993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:46.753993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:46.753993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:46.753993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:46.753993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:46.753993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:46.754915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:46.754915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:46.754915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:46.754915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:46.754915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:46.754915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:46.754915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:46.754915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:46.754915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:46.754915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:46.754915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:46.754915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:46.754915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:46.754915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:46.754915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:46.754915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:46.755784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:46.755784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:46.755784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:46.755784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:46.755784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:46.755784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:46.755784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:46.755784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:46.755784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:46.755784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:46.755784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:07:46.755784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:46.755784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 228/446] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:07:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:46.756465 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a02d36-085a-4eca-b700-cc33eba3e4cc None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:46.787898 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:46.832568 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9f2815-742a-4fc0-8391-b9b82cc83624 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:46.856759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a02d36-085a-4eca-b700-cc33eba3e4cc None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:46.865877 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85302a51-186a-49e0-a8ea-53f382026087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:46.866540 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85302a51-186a-49e0-a8ea-53f382026087 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:46.866883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85302a51-186a-49e0-a8ea-53f382026087 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:46.867154 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85302a51-186a-49e0-a8ea-53f382026087 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:46.868029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85302a51-186a-49e0-a8ea-53f382026087 None admin] RBAC: Adding query filter params (group.id=14581f2851d6447b8f3c0df3db37ca66, role.id=ca91f64e7a46431c8aa4285c4620f216, scope.project.id=c05a3987827c46a69dd1ab7b7f4b913b) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:46.868339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85302a51-186a-49e0-a8ea-53f382026087 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:46.869789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85302a51-186a-49e0-a8ea-53f382026087 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:46.875922 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 219/447] 10.222.0.81 () {56 vars in 1376 bytes} [Tue Jun 7 14:07:46 2022] GET /identity/v3/role_assignments?group.id=14581f2851d6447b8f3c0df3db37ca66&scope.project.id=c05a3987827c46a69dd1ab7b7f4b913b&role.id=ca91f64e7a46431c8aa4285c4620f216 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:46.876748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85302a51-186a-49e0-a8ea-53f382026087 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:46.900504 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:46.908210 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131d93e-d7d2-4401-8d15-04fea6919634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:46.909104 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131d93e-d7d2-4401-8d15-04fea6919634 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:46.909395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131d93e-d7d2-4401-8d15-04fea6919634 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:46.909665 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131d93e-d7d2-4401-8d15-04fea6919634 None admin] PATH_INFO: `/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:46.911056 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c131d93e-d7d2-4401-8d15-04fea6919634 None admin] RBAC: Authorizing `identity:get_project(project_id=c05a3987827c46a69dd1ab7b7f4b913b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:46.912574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c131d93e-d7d2-4401-8d15-04fea6919634 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:46.918800 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 229/448] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:46 2022] GET /identity/v3/projects/c05a3987827c46a69dd1ab7b7f4b913b => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:46.920083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131d93e-d7d2-4401-8d15-04fea6919634 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:46.999233 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85302a51-186a-49e0-a8ea-53f382026087 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:47.022852 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:47.045985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8cb36f-ba3f-4de3-9ee1-667cf9a20b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:47.046685 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cb36f-ba3f-4de3-9ee1-667cf9a20b94 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:47.046954 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cb36f-ba3f-4de3-9ee1-667cf9a20b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:47.047249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cb36f-ba3f-4de3-9ee1-667cf9a20b94 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:47.048020 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8cb36f-ba3f-4de3-9ee1-667cf9a20b94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:47.048324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8cb36f-ba3f-4de3-9ee1-667cf9a20b94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:47.049724 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8cb36f-ba3f-4de3-9ee1-667cf9a20b94 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:47.064861 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 220/449] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:47.066038 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8cb36f-ba3f-4de3-9ee1-667cf9a20b94 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:47.177012 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131d93e-d7d2-4401-8d15-04fea6919634 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:47.187020 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8cb36f-ba3f-4de3-9ee1-667cf9a20b94 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:47.218998 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4acd5-c59f-4ac5-8aba-7347944b3ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:47.219903 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4acd5-c59f-4ac5-8aba-7347944b3ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:47.220181 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4acd5-c59f-4ac5-8aba-7347944b3ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:47.220464 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4acd5-c59f-4ac5-8aba-7347944b3ff5 None admin] PATH_INFO: `/v3/groups/14581f2851d6447b8f3c0df3db37ca66` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:47.221618 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4acd5-c59f-4ac5-8aba-7347944b3ff5 None admin] RBAC: Authorizing `identity:get_group(group_id=14581f2851d6447b8f3c0df3db37ca66)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:47.223050 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4acd5-c59f-4ac5-8aba-7347944b3ff5 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:47.224263 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3fa64e-6664-4067-8b79-5c327c9bbf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:47.224727 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 230/450] 10.222.0.81 () {56 vars in 1180 bytes} [Tue Jun 7 14:07:46 2022] GET /identity/v3/groups/14581f2851d6447b8f3c0df3db37ca66 => generated 226 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:47.224922 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fa64e-6664-4067-8b79-5c327c9bbf3b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:47.225204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fa64e-6664-4067-8b79-5c327c9bbf3b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:47.225479 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fa64e-6664-4067-8b79-5c327c9bbf3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:47.231174 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4acd5-c59f-4ac5-8aba-7347944b3ff5 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:47.286387 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:47.310909 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3fa64e-6664-4067-8b79-5c327c9bbf3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:47.312396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3fa64e-6664-4067-8b79-5c327c9bbf3b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:47.317478 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:47.353304 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4acd5-c59f-4ac5-8aba-7347944b3ff5 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:47.387547 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e8623-915e-4333-9aa8-48488e3c7a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:47.388299 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e8623-915e-4333-9aa8-48488e3c7a4f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:47.388584 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e8623-915e-4333-9aa8-48488e3c7a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:47.388861 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e8623-915e-4333-9aa8-48488e3c7a4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:47.389688 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3e8623-915e-4333-9aa8-48488e3c7a4f None admin] RBAC: Adding query filter params (group.id=14581f2851d6447b8f3c0df3db37ca66, role.id=6d02db1fe03b447c9e519f4f2aa16574, scope.project.id=c05a3987827c46a69dd1ab7b7f4b913b) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:47.390003 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3e8623-915e-4333-9aa8-48488e3c7a4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:47.391579 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3e8623-915e-4333-9aa8-48488e3c7a4f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:47.397261 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 231/451] 10.222.0.81 () {56 vars in 1376 bytes} [Tue Jun 7 14:07:47 2022] GET /identity/v3/role_assignments?group.id=14581f2851d6447b8f3c0df3db37ca66&scope.project.id=c05a3987827c46a69dd1ab7b7f4b913b&role.id=6d02db1fe03b447c9e519f4f2aa16574 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:47.411666 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 221/452] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:07:47.418081 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e8623-915e-4333-9aa8-48488e3c7a4f None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:47.525057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e8623-915e-4333-9aa8-48488e3c7a4f None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:47.554272 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:47.560724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8a99b-5959-41eb-87af-e5dfa19ff490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:47.561615 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8a99b-5959-41eb-87af-e5dfa19ff490 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:47.561900 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8a99b-5959-41eb-87af-e5dfa19ff490 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:47.562178 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8a99b-5959-41eb-87af-e5dfa19ff490 None admin] PATH_INFO: `/v3/projects/50e31e6b5bc94b188158e931e4ee4076` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:47.563473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad8a99b-5959-41eb-87af-e5dfa19ff490 None admin] RBAC: Authorizing `identity:get_project(project_id=50e31e6b5bc94b188158e931e4ee4076)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:47.564931 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad8a99b-5959-41eb-87af-e5dfa19ff490 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:47.567324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 232/453] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:07:47 2022] GET /identity/v3/projects/50e31e6b5bc94b188158e931e4ee4076 => generated 344 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:47.573266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3fa64e-6664-4067-8b79-5c327c9bbf3b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:47.656307 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:47.677501 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3fa64e-6664-4067-8b79-5c327c9bbf3b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:47.712215 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16609de0-46d6-4f1b-8717-bb7415f22ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:47.713070 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16609de0-46d6-4f1b-8717-bb7415f22ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:47.713356 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16609de0-46d6-4f1b-8717-bb7415f22ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:47.713626 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16609de0-46d6-4f1b-8717-bb7415f22ce6 None admin] PATH_INFO: `/v3/groups/ee975d59db9348eab48f086cff604a7e` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:47.714803 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16609de0-46d6-4f1b-8717-bb7415f22ce6 None admin] RBAC: Authorizing `identity:get_group(group_id=ee975d59db9348eab48f086cff604a7e)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:47.716279 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16609de0-46d6-4f1b-8717-bb7415f22ce6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:47.717901 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 222/454] 10.222.0.81 () {56 vars in 1180 bytes} [Tue Jun 7 14:07:47 2022] GET /identity/v3/groups/ee975d59db9348eab48f086cff604a7e => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:47.723753 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8a99b-5959-41eb-87af-e5dfa19ff490 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:47.828234 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8a99b-5959-41eb-87af-e5dfa19ff490 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:47.859653 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b17ec-5d18-443a-8088-e7dce45d3785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:47.860296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b17ec-5d18-443a-8088-e7dce45d3785 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:47.860576 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b17ec-5d18-443a-8088-e7dce45d3785 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:47.860848 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b17ec-5d18-443a-8088-e7dce45d3785 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:47.861663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b17ec-5d18-443a-8088-e7dce45d3785 None admin] RBAC: Adding query filter params (group.id=ee975d59db9348eab48f086cff604a7e, role.id=82bbe3651b9b4f378fd150d93357b2b1, scope.project.id=50e31e6b5bc94b188158e931e4ee4076) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:47.861973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b17ec-5d18-443a-8088-e7dce45d3785 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:47.863412 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b17ec-5d18-443a-8088-e7dce45d3785 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:47.869588 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 233/455] 10.222.0.81 () {56 vars in 1376 bytes} [Tue Jun 7 14:07:47 2022] GET /identity/v3/role_assignments?group.id=ee975d59db9348eab48f086cff604a7e&scope.project.id=50e31e6b5bc94b188158e931e4ee4076&role.id=82bbe3651b9b4f378fd150d93357b2b1 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:50.493277 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58407d92-9ec3-4762-872f-fbd3df817ebe None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:50.493792 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58407d92-9ec3-4762-872f-fbd3df817ebe None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:50.493792 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58407d92-9ec3-4762-872f-fbd3df817ebe None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:50.495498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 223/456] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:50.500785 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d648a98-4dd7-4c5f-b9ff-8e120b3854e1 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:50.501192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d648a98-4dd7-4c5f-b9ff-8e120b3854e1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:50.501357 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d648a98-4dd7-4c5f-b9ff-8e120b3854e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:50.525177 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-2d648a98-4dd7-4c5f-b9ff-8e120b3854e1 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:50.526349 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2d648a98-4dd7-4c5f-b9ff-8e120b3854e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:50.606317 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:50.617693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 234/457] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:50.652600 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58407d92-9ec3-4762-872f-fbd3df817ebe None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:50.795313 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43fad4-cefd-4b32-97e0-5c77274f7e02 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:50.795802 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43fad4-cefd-4b32-97e0-5c77274f7e02 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:50.796233 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43fad4-cefd-4b32-97e0-5c77274f7e02 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:50.797952 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 235/458] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:50.803245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cb9ae-678b-49e2-9de5-04d9be636269 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:50.803800 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cb9ae-678b-49e2-9de5-04d9be636269 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:50.804807 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cb9ae-678b-49e2-9de5-04d9be636269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:50.808842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58407d92-9ec3-4762-872f-fbd3df817ebe None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:50.823968 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-c88cb9ae-678b-49e2-9de5-04d9be636269 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:50.825743 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c88cb9ae-678b-49e2-9de5-04d9be636269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:50.859597 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2bccb-1328-47e0-821c-267d4262b4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:50.860822 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2bccb-1328-47e0-821c-267d4262b4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:50.861244 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2bccb-1328-47e0-821c-267d4262b4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:50.861652 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2bccb-1328-47e0-821c-267d4262b4d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:50.876520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c2bccb-1328-47e0-821c-267d4262b4d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:50.877992 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c2bccb-1328-47e0-821c-267d4262b4d5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:50.890798 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:50.904978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-36c2bccb-1328-47e0-821c-267d4262b4d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:07:50.904978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:50.904978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:50.904978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:50.904978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:50.904978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:50.904978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:50.904978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:50.904978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:50.904978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:50.904978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:50.904978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:50.904978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:07:50.904978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:07:50.904978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:07:50.906068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:07:50.906068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:50.906068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:50.906068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:50.906068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:50.906068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:50.906068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:50.906068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:50.906068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:50.906068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:50.906068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:50.906068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:50.906068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:50.906068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:50.906068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:50.906068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:50.906068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:50.907142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:07:50.907142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:07:50.907142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:07:50.907142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:50.909466 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 224/459] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:07:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:50.910732 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babf843-1ba0-441a-9920-8e8839cb9683 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:50.911016 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babf843-1ba0-441a-9920-8e8839cb9683 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:50.911885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 236/460] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:07:50.915347 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babf843-1ba0-441a-9920-8e8839cb9683 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:50.916981 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 225/461] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:50.917185 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88cb9ae-678b-49e2-9de5-04d9be636269 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:50.928659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515bafc1-8e71-4cc2-8025-54be1ee10c9d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:50.929865 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515bafc1-8e71-4cc2-8025-54be1ee10c9d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:50.930525 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515bafc1-8e71-4cc2-8025-54be1ee10c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:50.961121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-515bafc1-8e71-4cc2-8025-54be1ee10c9d None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:50.963280 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-515bafc1-8e71-4cc2-8025-54be1ee10c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:51.044334 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88cb9ae-678b-49e2-9de5-04d9be636269 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:51.059342 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 226/462] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:51.059342 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515bafc1-8e71-4cc2-8025-54be1ee10c9d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:51.091052 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:51.108739 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891bfbe8-2f6e-4d3d-80ab-8c44d584a5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:51.109692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891bfbe8-2f6e-4d3d-80ab-8c44d584a5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:51.110150 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891bfbe8-2f6e-4d3d-80ab-8c44d584a5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:51.110568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891bfbe8-2f6e-4d3d-80ab-8c44d584a5f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:51.111485 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891bfbe8-2f6e-4d3d-80ab-8c44d584a5f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:51.111958 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891bfbe8-2f6e-4d3d-80ab-8c44d584a5f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:51.113497 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891bfbe8-2f6e-4d3d-80ab-8c44d584a5f7 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:51.144846 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 237/463] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:51.144846 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891bfbe8-2f6e-4d3d-80ab-8c44d584a5f7 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:51.204708 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515bafc1-8e71-4cc2-8025-54be1ee10c9d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:51.244736 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:51.251793 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb265574-189f-4d8a-9c30-1e54f27d5ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:51.252893 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb265574-189f-4d8a-9c30-1e54f27d5ddc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:51.253786 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb265574-189f-4d8a-9c30-1e54f27d5ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:51.254204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb265574-189f-4d8a-9c30-1e54f27d5ddc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:51.269271 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb265574-189f-4d8a-9c30-1e54f27d5ddc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:51.271659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb265574-189f-4d8a-9c30-1e54f27d5ddc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:51.286806 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891bfbe8-2f6e-4d3d-80ab-8c44d584a5f7 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:51.299015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-eb265574-189f-4d8a-9c30-1e54f27d5ddc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:07:51.299015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:51.299015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:51.299015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:51.299015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:51.299015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:51.299015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:51.299015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:51.299015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:51.299015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:51.299015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:51.299015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:51.299015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:07:51.299015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:07:51.299015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:07:51.299970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:07:51.299970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:51.299970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:51.299970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:51.299970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:51.299970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:51.299970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:51.299970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:51.299970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:51.299970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:51.299970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:51.299970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:51.299970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:51.299970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:51.299970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:51.299970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:51.299970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:51.300811 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:07:51.300811 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:07:51.300811 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:07:51.300811 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:51.308672 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 227/464] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:07:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:51.308672 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb265574-189f-4d8a-9c30-1e54f27d5ddc None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:51.333150 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6e5eb-a599-40ea-b7fa-a114d2e37a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:51.334405 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6e5eb-a599-40ea-b7fa-a114d2e37a94 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:51.334876 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6e5eb-a599-40ea-b7fa-a114d2e37a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:51.335401 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6e5eb-a599-40ea-b7fa-a114d2e37a94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:51.346768 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c6e5eb-a599-40ea-b7fa-a114d2e37a94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:51.348487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c6e5eb-a599-40ea-b7fa-a114d2e37a94 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:51.369046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-c7c6e5eb-a599-40ea-b7fa-a114d2e37a94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:07:51.369046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:51.369046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:51.369046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:51.369046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:51.369046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:51.369046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:51.369046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:51.369046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:51.369046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:51.369046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:51.369046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:51.369046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:07:51.369046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:07:51.369046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:07:51.369935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:07:51.369935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:51.369935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:51.369935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:51.369935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:51.369935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:51.369935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:51.369935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:51.369935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:51.369935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:51.369935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:51.369935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:51.369935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:51.369935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:51.369935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:51.369935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:51.369935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:51.370769 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:07:51.370769 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:07:51.370769 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:07:51.370769 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:51.373364 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 238/465] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:07:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:51.374430 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6e5eb-a599-40ea-b7fa-a114d2e37a94 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:51.409392 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:51.418118 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb265574-189f-4d8a-9c30-1e54f27d5ddc None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:51.453678 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7bfe56-4c21-49dd-8ab8-70e72e938b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:51.454347 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bfe56-4c21-49dd-8ab8-70e72e938b5e None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:51.454628 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bfe56-4c21-49dd-8ab8-70e72e938b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:51.454903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bfe56-4c21-49dd-8ab8-70e72e938b5e None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:51.455642 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7bfe56-4c21-49dd-8ab8-70e72e938b5e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:51.457067 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7bfe56-4c21-49dd-8ab8-70e72e938b5e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:51.478559 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 228/466] 10.222.0.81 () {60 vars in 1132 bytes} [Tue Jun 7 14:07:51 2022] POST /identity/v3/users => generated 246 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:07:51.480134 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6e5eb-a599-40ea-b7fa-a114d2e37a94 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:51.480529 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7bfe56-4c21-49dd-8ab8-70e72e938b5e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:51.538022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda8c45-18da-4b79-8366-2881133dc755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:51.540700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:51.541505 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda8c45-18da-4b79-8366-2881133dc755 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:51.542099 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda8c45-18da-4b79-8366-2881133dc755 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:51.542569 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda8c45-18da-4b79-8366-2881133dc755 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:51.544578 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda8c45-18da-4b79-8366-2881133dc755 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:51.545244 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda8c45-18da-4b79-8366-2881133dc755 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:51.547027 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda8c45-18da-4b79-8366-2881133dc755 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:51.562558 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 239/467] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:51.565113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2243171-7a9a-4e01-b956-a4150cbaaa3f None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:51.565831 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2243171-7a9a-4e01-b956-a4150cbaaa3f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:51.566552 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2243171-7a9a-4e01-b956-a4150cbaaa3f None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:51.573234 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 240/468] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:51 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:51.574780 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2243171-7a9a-4e01-b956-a4150cbaaa3f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:51.634689 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7bfe56-4c21-49dd-8ab8-70e72e938b5e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:51.677121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064ae66-5db1-4678-9393-12ff187f2080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:51.677803 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064ae66-5db1-4678-9393-12ff187f2080 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:51.678081 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064ae66-5db1-4678-9393-12ff187f2080 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:51.678350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064ae66-5db1-4678-9393-12ff187f2080 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:51.679661 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9064ae66-5db1-4678-9393-12ff187f2080 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:51.679976 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9064ae66-5db1-4678-9393-12ff187f2080 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:51.681480 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9064ae66-5db1-4678-9393-12ff187f2080 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:51.689513 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:51.698386 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 229/469] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:51.701585 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a5620-589d-4023-aee0-5bd94b4f5464 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:51.702416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a5620-589d-4023-aee0-5bd94b4f5464 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:51.703615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a5620-589d-4023-aee0-5bd94b4f5464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:51.704065 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2243171-7a9a-4e01-b956-a4150cbaaa3f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:51.721679 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-537a5620-589d-4023-aee0-5bd94b4f5464 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:51.723042 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-537a5620-589d-4023-aee0-5bd94b4f5464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:51.743427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f6105-79a2-45a4-9166-fc35a1d19802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:51.744104 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f6105-79a2-45a4-9166-fc35a1d19802 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:51.744382 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f6105-79a2-45a4-9166-fc35a1d19802 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:51.744656 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f6105-79a2-45a4-9166-fc35a1d19802 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:51.745366 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2f6105-79a2-45a4-9166-fc35a1d19802 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:51.747582 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2f6105-79a2-45a4-9166-fc35a1d19802 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:51.776380 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 241/470] 10.222.0.81 () {60 vars in 1132 bytes} [Tue Jun 7 14:07:51 2022] POST /identity/v3/users => generated 248 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:07:51.777196 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f6105-79a2-45a4-9166-fc35a1d19802 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:51.831242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 230/471] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:51.859702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a5620-589d-4023-aee0-5bd94b4f5464 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:51.923845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:51.973528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f6105-79a2-45a4-9166-fc35a1d19802 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:52.029509 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491417e9-fd08-4d78-bd80-d15f8766ae16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:52.030458 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491417e9-fd08-4d78-bd80-d15f8766ae16 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:52.030740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491417e9-fd08-4d78-bd80-d15f8766ae16 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:52.031015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491417e9-fd08-4d78-bd80-d15f8766ae16 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:52.031908 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491417e9-fd08-4d78-bd80-d15f8766ae16 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:52.033666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491417e9-fd08-4d78-bd80-d15f8766ae16 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:52.056283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a5620-589d-4023-aee0-5bd94b4f5464 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:52.067298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 242/472] 10.222.0.81 () {60 vars in 1132 bytes} [Tue Jun 7 14:07:51 2022] POST /identity/v3/users => generated 248 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:07:52.114746 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b45c8-00b7-499b-aedd-688641046de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:52.115653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b45c8-00b7-499b-aedd-688641046de9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:52.116018 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b45c8-00b7-499b-aedd-688641046de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:52.116295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b45c8-00b7-499b-aedd-688641046de9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:52.129732 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b45c8-00b7-499b-aedd-688641046de9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:52.131390 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b45c8-00b7-499b-aedd-688641046de9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:52.151903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-919b45c8-00b7-499b-aedd-688641046de9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:07:52.151903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:52.151903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:52.151903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:52.151903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:52.151903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:52.151903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:52.151903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:52.151903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:52.151903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:52.151903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:52.151903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:52.151903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:07:52.151903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:07:52.151903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:07:52.152941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:07:52.152941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:52.152941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:52.152941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:52.152941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:52.152941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:52.152941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:52.152941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:52.152941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:52.152941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:52.152941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:52.152941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:52.152941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:52.152941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:52.152941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:52.152941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:52.152941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:52.153933 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:07:52.153933 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:07:52.153933 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:07:52.153933 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:52.153933 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 231/473] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:07:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:52.161316 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491417e9-fd08-4d78-bd80-d15f8766ae16 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:52.179722 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:52.281001 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491417e9-fd08-4d78-bd80-d15f8766ae16 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:52.306928 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd347a-8209-4410-9c50-20e2b4ec9a0e None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:52.307266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd347a-8209-4410-9c50-20e2b4ec9a0e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:52.307517 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd347a-8209-4410-9c50-20e2b4ec9a0e None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:52.309789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 232/474] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:52.321130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84978b-c7a0-41fb-a1d7-8c36d460a150 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:52.321585 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84978b-c7a0-41fb-a1d7-8c36d460a150 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:52.321876 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84978b-c7a0-41fb-a1d7-8c36d460a150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:52.327466 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062bf75f-f112-4359-8eff-7caae11bb252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:52.328395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062bf75f-f112-4359-8eff-7caae11bb252 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:52.328871 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062bf75f-f112-4359-8eff-7caae11bb252 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:52.329339 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062bf75f-f112-4359-8eff-7caae11bb252 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:52.330266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062bf75f-f112-4359-8eff-7caae11bb252 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:52.330766 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062bf75f-f112-4359-8eff-7caae11bb252 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:52.332446 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062bf75f-f112-4359-8eff-7caae11bb252 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:52.346349 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 243/475] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:52.348190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1f84978b-c7a0-41fb-a1d7-8c36d460a150 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:52.349224 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1f84978b-c7a0-41fb-a1d7-8c36d460a150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:52.353595 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062bf75f-f112-4359-8eff-7caae11bb252 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:52.429869 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 233/476] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:52.445320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:52.452549 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84978b-c7a0-41fb-a1d7-8c36d460a150 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:52.466697 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:52.496320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062bf75f-f112-4359-8eff-7caae11bb252 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:52.544398 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f9b4d-3c97-4fe9-8bb7-1e772a9f7bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:52.545179 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f9b4d-3c97-4fe9-8bb7-1e772a9f7bef None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:52.545386 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f9b4d-3c97-4fe9-8bb7-1e772a9f7bef None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:52.545664 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f9b4d-3c97-4fe9-8bb7-1e772a9f7bef None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:52.546382 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f9b4d-3c97-4fe9-8bb7-1e772a9f7bef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:52.547863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f9b4d-3c97-4fe9-8bb7-1e772a9f7bef None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:52.573304 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 244/477] 10.222.0.81 () {60 vars in 1132 bytes} [Tue Jun 7 14:07:52 2022] POST /identity/v3/users => generated 249 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:07:52.580339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84978b-c7a0-41fb-a1d7-8c36d460a150 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:52.648990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7912ff8-6ab2-4eec-bc7a-b85ce61dda11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:52.649703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7912ff8-6ab2-4eec-bc7a-b85ce61dda11 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:52.649988 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7912ff8-6ab2-4eec-bc7a-b85ce61dda11 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:52.650270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7912ff8-6ab2-4eec-bc7a-b85ce61dda11 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:52.651055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7912ff8-6ab2-4eec-bc7a-b85ce61dda11 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:52.652486 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7912ff8-6ab2-4eec-bc7a-b85ce61dda11 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:52.665832 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.sql.core [None req-c7912ff8-6ab2-4eec-bc7a-b85ce61dda11 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 07 14:07:52.665832 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 07 14:07:52.665832 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [parameters: {'id': '2359189a4f1b442a8f0aff4638d3fc72', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 07 14:07:52.665832 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80233) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 07 14:07:52.682805 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-c7912ff8-6ab2-4eec-bc7a-b85ce61dda11 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 07 14:07:52.682805 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:52.682805 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 14:07:52.682805 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 14:07:52.682805 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 14:07:52.682805 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 14:07:52.682805 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 14:07:52.682805 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 14:07:52.682805 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 14:07:52.682805 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application conn.query(q) Jun 07 14:07:52.682805 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 14:07:52.682805 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 14:07:52.682805 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 14:07:52.682805 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application result.read() Jun 07 14:07:52.682805 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 14:07:52.683757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 14:07:52.683757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 14:07:52.683757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 14:07:52.683757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 14:07:52.683757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 14:07:52.683757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 14:07:52.683757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 14:07:52.683757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 07 14:07:52.683757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:52.683757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 07 14:07:52.683757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:52.683757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:52.683757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 07 14:07:52.683757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 07 14:07:52.683757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 07 14:07:52.683757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return ref.to_dict() Jun 07 14:07:52.683757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 14:07:52.683757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application next(self.gen) Jun 07 14:07:52.684721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 07 14:07:52.684721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application yield resource Jun 07 14:07:52.684721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 14:07:52.684721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application next(self.gen) Jun 07 14:07:52.684721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 07 14:07:52.684721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.session.rollback() Jun 07 14:07:52.684721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 07 14:07:52.684721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.force_reraise() Jun 07 14:07:52.684721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 07 14:07:52.684721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise self.value Jun 07 14:07:52.684721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 07 14:07:52.684721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 07 14:07:52.684721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 07 14:07:52.684721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application session.commit() Jun 07 14:07:52.684721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 07 14:07:52.684721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 07 14:07:52.684721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 07 14:07:52.685591 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._prepare_impl() Jun 07 14:07:52.685591 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 07 14:07:52.685591 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.session.flush() Jun 07 14:07:52.685591 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 07 14:07:52.685591 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._flush(objects) Jun 07 14:07:52.685591 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 07 14:07:52.685591 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 07 14:07:52.685591 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 07 14:07:52.685591 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application compat.raise_( Jun 07 14:07:52.685591 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 14:07:52.685591 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception Jun 07 14:07:52.685591 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 07 14:07:52.685591 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application flush_context.execute() Jun 07 14:07:52.685591 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 07 14:07:52.685591 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rec.execute(self) Jun 07 14:07:52.685591 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 07 14:07:52.685591 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 07 14:07:52.686490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 07 14:07:52.686490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 07 14:07:52.686490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 07 14:07:52.686490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 07 14:07:52.686490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 07 14:07:52.686490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 07 14:07:52.686490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 07 14:07:52.686490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 07 14:07:52.686490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 07 14:07:52.686490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 07 14:07:52.686490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 07 14:07:52.686490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 07 14:07:52.686490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 07 14:07:52.686490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 07 14:07:52.686490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 14:07:52.686490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception Jun 07 14:07:52.688619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 14:07:52.688619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 14:07:52.688619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 14:07:52.688619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 14:07:52.688619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 14:07:52.688619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 14:07:52.688619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 14:07:52.688619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application conn.query(q) Jun 07 14:07:52.688619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 14:07:52.688619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 14:07:52.688619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 14:07:52.688619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application result.read() Jun 07 14:07:52.688619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 14:07:52.688619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 14:07:52.688619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 14:07:52.688619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 14:07:52.690553 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 14:07:52.690553 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 14:07:52.690553 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 14:07:52.690553 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 14:07:52.690553 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 07 14:07:52.690553 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 07 14:07:52.690553 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [parameters: {'id': '2359189a4f1b442a8f0aff4638d3fc72', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 07 14:07:52.690553 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 07 14:07:52.690553 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:52.690553 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:07:52.690553 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:52.690553 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:52.690553 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:52.690553 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:52.690553 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:52.690553 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:52.694952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:52.694952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:52.694952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:52.694952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:52.694952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:52.694952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:52.694952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 07 14:07:52.694952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 07 14:07:52.694952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:52.694952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:52.694952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 07 14:07:52.694952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 07 14:07:52.694952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 07 14:07:52.694952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 07 14:07:52.694952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 07 14:07:52.694952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:52.697057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 234/478] 10.222.0.81 () {60 vars in 1131 bytes} [Tue Jun 7 14:07:52 2022] POST /identity/v3/roles => generated 144 bytes in 232 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:52.697193 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f9b4d-3c97-4fe9-8bb7-1e772a9f7bef None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:52.828426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f9b4d-3c97-4fe9-8bb7-1e772a9f7bef None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:52.853015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:52.865075 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91112540-48e4-4dfe-9974-ae68c5fc0478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:52.865894 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91112540-48e4-4dfe-9974-ae68c5fc0478 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:52.866609 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91112540-48e4-4dfe-9974-ae68c5fc0478 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:52.867323 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91112540-48e4-4dfe-9974-ae68c5fc0478 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:52.878601 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91112540-48e4-4dfe-9974-ae68c5fc0478 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:52.887634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-91112540-48e4-4dfe-9974-ae68c5fc0478 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:07:52.887634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:52.887634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:52.887634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:52.887634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:52.887634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:52.887634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:52.887634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:52.887634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:52.887634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:52.887634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:52.887634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:52.887634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:52.887634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:52.887634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:52.887634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:52.888749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:52.888749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:52.888749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:52.888749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:52.888749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:52.888749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:52.888749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:52.888749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:52.888749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:52.888749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:52.888749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:52.888749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:52.888749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:52.888749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:52.888749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:52.888749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:52.889683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:52.889683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:52.889683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:52.889683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:52.889683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:52.889683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:52.889683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:52.889683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:52.889683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:52.889683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:52.889683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:07:52.889683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:52.889683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 245/479] 10.222.0.81 () {56 vars in 1114 bytes} [Tue Jun 7 14:07:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:52.896409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7912ff8-6ab2-4eec-bc7a-b85ce61dda11 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:52.985447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:53.014458 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7912ff8-6ab2-4eec-bc7a-b85ce61dda11 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:53.053385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17c162-af96-4b14-8f29-d7b478cbc249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:53.054486 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17c162-af96-4b14-8f29-d7b478cbc249 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:53.055005 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17c162-af96-4b14-8f29-d7b478cbc249 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:53.055499 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17c162-af96-4b14-8f29-d7b478cbc249 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:53.056452 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd17c162-af96-4b14-8f29-d7b478cbc249 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:53.056963 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd17c162-af96-4b14-8f29-d7b478cbc249 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:53.058558 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd17c162-af96-4b14-8f29-d7b478cbc249 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:53.068554 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 235/480] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:07:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:54.505337 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ba924-37d9-4313-b79e-7444533a1426 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:54.505960 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ba924-37d9-4313-b79e-7444533a1426 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:54.505960 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ba924-37d9-4313-b79e-7444533a1426 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:54.507552 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 246/481] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:54.513068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c05ba4-fc20-4a76-a0f5-1de2c98cd553 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:54.513491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c05ba4-fc20-4a76-a0f5-1de2c98cd553 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:54.513687 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c05ba4-fc20-4a76-a0f5-1de2c98cd553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:54.540477 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a0c05ba4-fc20-4a76-a0f5-1de2c98cd553 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:54.541666 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a0c05ba4-fc20-4a76-a0f5-1de2c98cd553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:54.639328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 236/482] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:54.645484 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e442ffa-2511-4f96-8675-f04287ecbbd4 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:54.645771 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e442ffa-2511-4f96-8675-f04287ecbbd4 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:54.646045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e442ffa-2511-4f96-8675-f04287ecbbd4 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:54.647719 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 247/483] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:54.652911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada5f0a-08fd-404f-bcdc-ba8e30ea38ed None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:54.653202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada5f0a-08fd-404f-bcdc-ba8e30ea38ed None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:54.653480 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada5f0a-08fd-404f-bcdc-ba8e30ea38ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:54.661065 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e442ffa-2511-4f96-8675-f04287ecbbd4 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:54.688223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9ada5f0a-08fd-404f-bcdc-ba8e30ea38ed None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:54.689419 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9ada5f0a-08fd-404f-bcdc-ba8e30ea38ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:54.800631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e442ffa-2511-4f96-8675-f04287ecbbd4 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:54.809734 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:54.828904 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 237/484] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:07:54.855938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74df153-f321-498b-a230-8ae079ee1951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:54.856748 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74df153-f321-498b-a230-8ae079ee1951 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:54.857029 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74df153-f321-498b-a230-8ae079ee1951 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:54.857301 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74df153-f321-498b-a230-8ae079ee1951 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:54.865071 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada5f0a-08fd-404f-bcdc-ba8e30ea38ed None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:54.869540 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74df153-f321-498b-a230-8ae079ee1951 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:54.878274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-f74df153-f321-498b-a230-8ae079ee1951 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:07:54.878274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:54.878274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:54.878274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:54.878274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:54.878274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:54.878274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:54.878274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:54.878274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:54.878274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:54.878274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:54.878274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:54.878274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:54.878274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:54.878274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:54.878274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:54.879189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:54.879189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:54.879189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:54.879189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:54.879189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:54.879189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:54.879189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:54.879189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:54.879189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:54.879189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:54.879189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:54.879189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:54.879189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:54.879189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:54.879189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:54.879189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:54.880105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:54.880105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:54.880105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:54.880105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:54.880105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:54.880105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:54.880105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:54.880105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:54.880105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:54.880105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:54.880105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:07:54.880105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:54.880105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 248/485] 10.222.0.81 () {56 vars in 1102 bytes} [Tue Jun 7 14:07:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:54.888682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74df153-f321-498b-a230-8ae079ee1951 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:54.939661 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:54.969591 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada5f0a-08fd-404f-bcdc-ba8e30ea38ed None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:55.003649 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74df153-f321-498b-a230-8ae079ee1951 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:55.005786 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9855d2-df21-4cc3-b999-d434983d3614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:55.006592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9855d2-df21-4cc3-b999-d434983d3614 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:55.006872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9855d2-df21-4cc3-b999-d434983d3614 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:55.007250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9855d2-df21-4cc3-b999-d434983d3614 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:55.018579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9855d2-df21-4cc3-b999-d434983d3614 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:55.026653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-8b9855d2-df21-4cc3-b999-d434983d3614 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:07:55.026653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:55.026653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:55.026653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:55.026653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:55.026653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:55.026653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:55.026653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:55.026653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:55.026653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:55.026653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:55.026653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:55.026653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:55.026653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:55.026653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:55.026653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:55.027589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:55.027589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:55.027589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:55.027589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:55.027589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:55.027589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:55.027589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:55.027589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:55.027589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:55.027589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:55.027589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:55.027589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:55.027589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:55.027589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:55.027589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:55.027589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:55.028418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:55.028418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:55.028418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:55.028418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:55.028418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:55.028418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:55.028418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:55.028418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:55.028418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:55.028418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:55.028418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:07:55.028418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:55.028418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 238/486] 10.222.0.81 () {56 vars in 1102 bytes} [Tue Jun 7 14:07:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:55.029552 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd796b-9467-4536-9746-800055d13b9a None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:55.029719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd796b-9467-4536-9746-800055d13b9a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:55.029982 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd796b-9467-4536-9746-800055d13b9a None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:55.031613 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 239/487] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:55.034538 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd796b-9467-4536-9746-800055d13b9a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:55.048798 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea814a-9c22-4a76-8e1e-4b7c9f169acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:55.049750 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea814a-9c22-4a76-8e1e-4b7c9f169acd None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:55.050036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea814a-9c22-4a76-8e1e-4b7c9f169acd None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:55.050315 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea814a-9c22-4a76-8e1e-4b7c9f169acd None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:55.051411 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea814a-9c22-4a76-8e1e-4b7c9f169acd None admin] RBAC: Adding query filter params (name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:55.051672 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea814a-9c22-4a76-8e1e-4b7c9f169acd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:55.054749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea814a-9c22-4a76-8e1e-4b7c9f169acd None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:55.066568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 249/488] 10.222.0.81 () {56 vars in 1103 bytes} [Tue Jun 7 14:07:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:55.068459 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df0d45-1370-4a8f-af93-f68ebf8ec3b4 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:55.068754 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df0d45-1370-4a8f-af93-f68ebf8ec3b4 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:55.069061 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df0d45-1370-4a8f-af93-f68ebf8ec3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:55.096776 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-07df0d45-1370-4a8f-af93-f68ebf8ec3b4 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:55.097806 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:55.099798 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-07df0d45-1370-4a8f-af93-f68ebf8ec3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:55.146017 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd796b-9467-4536-9746-800055d13b9a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:55.189191 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1290e8-0a46-49b7-a483-325bb881c3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:55.190149 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1290e8-0a46-49b7-a483-325bb881c3c0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:55.190572 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1290e8-0a46-49b7-a483-325bb881c3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:55.190989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1290e8-0a46-49b7-a483-325bb881c3c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:55.191901 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1290e8-0a46-49b7-a483-325bb881c3c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:55.192337 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1290e8-0a46-49b7-a483-325bb881c3c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:55.193849 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1290e8-0a46-49b7-a483-325bb881c3c0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:55.199263 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 250/489] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:55.200838 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df0d45-1370-4a8f-af93-f68ebf8ec3b4 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:55.203583 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 240/490] 10.222.0.81 () {56 vars in 1103 bytes} [Tue Jun 7 14:07:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:55.209382 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1290e8-0a46-49b7-a483-325bb881c3c0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:55.244473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:55.315260 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1290e8-0a46-49b7-a483-325bb881c3c0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:55.319023 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df0d45-1370-4a8f-af93-f68ebf8ec3b4 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:55.333234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:55.353397 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4743d-e37e-49da-8229-99feca216ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:55.354596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4743d-e37e-49da-8229-99feca216ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:55.355098 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4743d-e37e-49da-8229-99feca216ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:55.355624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4743d-e37e-49da-8229-99feca216ce5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:55.360143 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d911e8b-1887-4d2c-8fbc-5f7151f31a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:55.361147 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d911e8b-1887-4d2c-8fbc-5f7151f31a58 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:55.361606 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d911e8b-1887-4d2c-8fbc-5f7151f31a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:55.362022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d911e8b-1887-4d2c-8fbc-5f7151f31a58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:55.369112 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d4743d-e37e-49da-8229-99feca216ce5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:55.371256 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d4743d-e37e-49da-8229-99feca216ce5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:55.375736 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d911e8b-1887-4d2c-8fbc-5f7151f31a58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:55.377234 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d911e8b-1887-4d2c-8fbc-5f7151f31a58 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:55.392157 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-42d4743d-e37e-49da-8229-99feca216ce5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:07:55.392157 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:55.392157 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:55.392157 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:55.392157 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:55.392157 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:55.392157 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:55.392157 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:55.392157 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:55.392157 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:55.392157 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:55.392157 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:55.392157 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:07:55.392157 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:07:55.392157 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:07:55.393103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:07:55.393103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:55.393103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:55.393103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:55.393103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:55.393103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:55.393103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:55.393103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:55.393103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:55.393103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:55.393103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:55.393103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:55.393103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:55.393103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:55.393103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:55.393103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:55.393103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:55.393956 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:07:55.393956 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:07:55.393956 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:07:55.393956 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:55.396386 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 241/491] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:07:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:07:55.397508 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4743d-e37e-49da-8229-99feca216ce5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:55.398296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-1d911e8b-1887-4d2c-8fbc-5f7151f31a58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:07:55.398296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:55.398296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:55.398296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:55.398296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:55.398296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:55.398296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:55.398296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:55.398296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:55.398296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:55.398296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:55.398296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:55.398296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:07:55.398296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:07:55.398296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:07:55.399116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:07:55.399116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:55.399116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:55.399116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:55.399116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:55.399116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:55.399116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:55.399116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:55.399116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:55.399116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:55.399116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:55.399116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:55.399116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:55.399116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:55.399116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:55.399116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:55.399116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:55.400117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:07:55.400117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:07:55.400117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:07:55.400117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:55.405150 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 251/492] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:07:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:55.405150 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d911e8b-1887-4d2c-8fbc-5f7151f31a58 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:55.499207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:55.505761 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4743d-e37e-49da-8229-99feca216ce5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:55.517476 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d911e8b-1887-4d2c-8fbc-5f7151f31a58 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:55.541734 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cc922-3cac-4133-802e-46996e3c1742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:55.543249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cc922-3cac-4133-802e-46996e3c1742 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:55.543752 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cc922-3cac-4133-802e-46996e3c1742 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:55.544234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cc922-3cac-4133-802e-46996e3c1742 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:55.555124 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ba489-90a3-4ed5-ab9e-e9dd80c6c185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:55.555817 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364cc922-3cac-4133-802e-46996e3c1742 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:55.555925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ba489-90a3-4ed5-ab9e-e9dd80c6c185 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:55.556214 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ba489-90a3-4ed5-ab9e-e9dd80c6c185 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:55.556487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ba489-90a3-4ed5-ab9e-e9dd80c6c185 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:55.557220 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ba489-90a3-4ed5-ab9e-e9dd80c6c185 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:55.557528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ba489-90a3-4ed5-ab9e-e9dd80c6c185 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:55.559004 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ba489-90a3-4ed5-ab9e-e9dd80c6c185 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:55.565594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-364cc922-3cac-4133-802e-46996e3c1742 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:07:55.565594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:55.565594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:55.565594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:55.565594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:55.565594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:55.565594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:55.565594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:55.565594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:55.565594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:55.565594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:55.565594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:55.565594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:55.565594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:55.565594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:55.565594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:55.566472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:55.566472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:55.566472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:55.566472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:55.566472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:55.566472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:55.566472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:55.566472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:55.566472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:55.566472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:55.566472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:55.566472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:55.566472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:55.566472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:55.566472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:55.566472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:55.567335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:55.567335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:55.567335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:55.567335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:55.567335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:55.567335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:55.567335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:55.567335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:55.567335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:55.567335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:55.567335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:07:55.567335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:55.570977 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 242/493] 10.222.0.81 () {56 vars in 1102 bytes} [Tue Jun 7 14:07:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:55.572050 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cc922-3cac-4133-802e-46996e3c1742 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:55.577841 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 252/494] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:55.578891 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ba489-90a3-4ed5-ab9e-e9dd80c6c185 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:55.608045 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:55.681798 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cc922-3cac-4133-802e-46996e3c1742 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:55.692531 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ba489-90a3-4ed5-ab9e-e9dd80c6c185 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:55.717384 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1b3e9-ca88-4cb6-b7bc-edc0761f6d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:55.718054 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1b3e9-ca88-4cb6-b7bc-edc0761f6d8a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:55.718337 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1b3e9-ca88-4cb6-b7bc-edc0761f6d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:55.718613 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1b3e9-ca88-4cb6-b7bc-edc0761f6d8a None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:55.719370 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1b3e9-ca88-4cb6-b7bc-edc0761f6d8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:55.719675 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1b3e9-ca88-4cb6-b7bc-edc0761f6d8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:55.721035 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1b3e9-ca88-4cb6-b7bc-edc0761f6d8a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:55.730627 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088921a-698f-44b1-ab7d-bf44b0da12fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:55.731328 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088921a-698f-44b1-ab7d-bf44b0da12fd None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:55.731611 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088921a-698f-44b1-ab7d-bf44b0da12fd None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:55.731883 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088921a-698f-44b1-ab7d-bf44b0da12fd None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:55.732635 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b088921a-698f-44b1-ab7d-bf44b0da12fd None admin] RBAC: Adding query filter params (name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:55.732938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b088921a-698f-44b1-ab7d-bf44b0da12fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:55.739012 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b088921a-698f-44b1-ab7d-bf44b0da12fd None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:55.739172 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 243/495] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:55.739172 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1b3e9-ca88-4cb6-b7bc-edc0761f6d8a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:55.744553 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 253/496] 10.222.0.81 () {56 vars in 1103 bytes} [Tue Jun 7 14:07:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:55.746450 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42374015-1743-4ae4-9e87-c5bb4ab5e50c None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:55.747527 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42374015-1743-4ae4-9e87-c5bb4ab5e50c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:55.747973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42374015-1743-4ae4-9e87-c5bb4ab5e50c None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:55.750050 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 254/497] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:55.751416 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42374015-1743-4ae4-9e87-c5bb4ab5e50c None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:55.764953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:55.838106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:55.847571 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1b3e9-ca88-4cb6-b7bc-edc0761f6d8a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:55.863666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42374015-1743-4ae4-9e87-c5bb4ab5e50c None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:55.887159 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49c9a3-108f-4425-a184-367ef77d8050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:55.888057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49c9a3-108f-4425-a184-367ef77d8050 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:55.888339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49c9a3-108f-4425-a184-367ef77d8050 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:55.888609 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49c9a3-108f-4425-a184-367ef77d8050 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:55.896059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49c9a3-108f-4425-a184-367ef77d8050 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:55.897503 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49c9a3-108f-4425-a184-367ef77d8050 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:55.901084 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d037e-658c-427b-875e-de8002e93b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:55.901884 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d037e-658c-427b-875e-de8002e93b18 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:55.902165 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d037e-658c-427b-875e-de8002e93b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:55.902433 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d037e-658c-427b-875e-de8002e93b18 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:55.910067 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d037e-658c-427b-875e-de8002e93b18 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:55.911532 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d037e-658c-427b-875e-de8002e93b18 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:55.914059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-4a49c9a3-108f-4425-a184-367ef77d8050 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 07 14:07:55.914059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:55.914059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:55.914059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:55.914059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:55.914059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:55.914059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:55.914059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:55.914059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:55.914059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:55.914059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:55.914059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:55.914059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:07:55.914059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:07:55.914059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:07:55.914059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:07:55.915076 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:55.915076 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:55.915076 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:07:55.915076 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:07:55.915076 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:07:55.915076 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:07:55.915076 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:55.915076 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:55.915076 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:55.915076 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:55.915076 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:55.915076 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:55.915076 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:55.915076 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:55.915076 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:55.915076 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:55.915076 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:55.915076 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:55.916748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:55.916748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:55.916748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:07:55.916748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:07:55.916748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:07:55.916748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:07:55.916748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:07:55.916748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:07:55.916748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 07 14:07:55.916748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:55.916748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 244/498] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:55.916748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49c9a3-108f-4425-a184-367ef77d8050 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:55.933511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-e93d037e-658c-427b-875e-de8002e93b18 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:07:55.933511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:55.933511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:55.933511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:55.933511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:55.933511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:55.933511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:55.933511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:55.933511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:55.933511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:55.933511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:55.933511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:55.933511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:07:55.933511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:07:55.933511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:07:55.933511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:07:55.934510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:55.934510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:55.934510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:07:55.934510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:07:55.934510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:07:55.934510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:07:55.934510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:55.934510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:55.934510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:55.934510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:55.934510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:55.934510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:55.934510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:55.934510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:55.934510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:55.934510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:55.934510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:55.934510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:55.935470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:55.935470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:55.935470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:07:55.935470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:07:55.935470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:07:55.935470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:07:55.935470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:07:55.935470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:07:55.935470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:07:55.935470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:55.935470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 255/499] 10.222.0.81 () {56 vars in 1093 bytes} [Tue Jun 7 14:07:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:55.936650 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165e98b-3fc1-4825-b384-5a91d39597ad None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:55.936650 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165e98b-3fc1-4825-b384-5a91d39597ad None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:55.936835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165e98b-3fc1-4825-b384-5a91d39597ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:55.954517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-8165e98b-3fc1-4825-b384-5a91d39597ad None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:55.955745 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8165e98b-3fc1-4825-b384-5a91d39597ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:56.022265 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:56.026930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49c9a3-108f-4425-a184-367ef77d8050 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:56.029898 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 256/500] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:56.030412 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165e98b-3fc1-4825-b384-5a91d39597ad None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:56.065110 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd649a4-c04c-4199-a493-e66c0f2544b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:56.065906 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd649a4-c04c-4199-a493-e66c0f2544b9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:56.066198 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd649a4-c04c-4199-a493-e66c0f2544b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:56.066462 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd649a4-c04c-4199-a493-e66c0f2544b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:56.077005 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd649a4-c04c-4199-a493-e66c0f2544b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:56.078454 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd649a4-c04c-4199-a493-e66c0f2544b9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:56.097324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-6fd649a4-c04c-4199-a493-e66c0f2544b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:07:56.097324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:56.097324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:56.097324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:56.097324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:56.097324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:56.097324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:56.097324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:56.097324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:56.097324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:56.097324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:56.097324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:56.097324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:07:56.097324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:07:56.097324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:07:56.098266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:07:56.098266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:56.098266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:56.098266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:56.098266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:56.098266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:56.098266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:56.098266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:56.098266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:56.098266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:56.098266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:56.098266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:56.098266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:56.098266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:56.098266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:56.098266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:56.098266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:56.099946 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:07:56.099946 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:07:56.099946 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:07:56.099946 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:56.099946 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 245/501] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:07:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:56.099946 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd649a4-c04c-4199-a493-e66c0f2544b9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:56.105864 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:56.143088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165e98b-3fc1-4825-b384-5a91d39597ad None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:56.177528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c0ee03-4334-481c-80aa-e70c0a521217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:56.178201 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0ee03-4334-481c-80aa-e70c0a521217 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:56.178479 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0ee03-4334-481c-80aa-e70c0a521217 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:56.178751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0ee03-4334-481c-80aa-e70c0a521217 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:56.179739 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c0ee03-4334-481c-80aa-e70c0a521217 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:56.180040 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c0ee03-4334-481c-80aa-e70c0a521217 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:56.181458 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c0ee03-4334-481c-80aa-e70c0a521217 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:56.205086 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd649a4-c04c-4199-a493-e66c0f2544b9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:56.217855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 257/502] 10.222.0.81 () {56 vars in 1137 bytes} [Tue Jun 7 14:07:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:56.218683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c0ee03-4334-481c-80aa-e70c0a521217 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:56.242590 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ccfa0-039d-4fcc-944b-9db62329fdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:56.243275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ccfa0-039d-4fcc-944b-9db62329fdd1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:56.243558 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ccfa0-039d-4fcc-944b-9db62329fdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:56.243829 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ccfa0-039d-4fcc-944b-9db62329fdd1 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:56.244658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ccfa0-039d-4fcc-944b-9db62329fdd1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:56.244960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ccfa0-039d-4fcc-944b-9db62329fdd1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:56.246665 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ccfa0-039d-4fcc-944b-9db62329fdd1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:56.276904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:56.325141 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c0ee03-4334-481c-80aa-e70c0a521217 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:56.360835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8607d91b-2843-4e6d-96fc-6a0703baad71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:56.361568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607d91b-2843-4e6d-96fc-6a0703baad71 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:56.361853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607d91b-2843-4e6d-96fc-6a0703baad71 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:56.362128 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607d91b-2843-4e6d-96fc-6a0703baad71 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:56.373583 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8607d91b-2843-4e6d-96fc-6a0703baad71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:07:56.381393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-8607d91b-2843-4e6d-96fc-6a0703baad71 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:07:56.381393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:56.381393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:56.381393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:56.381393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:56.381393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:56.381393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:56.381393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:56.381393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:56.381393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:56.381393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:56.381393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:56.381393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:07:56.381393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:07:56.381393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:07:56.381393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:07:56.382847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:07:56.382847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:07:56.382847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:07:56.382847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:07:56.382847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:56.382847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:56.382847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:56.382847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:56.382847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:56.382847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:56.382847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:56.382847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:56.382847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:56.382847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:56.382847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:56.382847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:56.383755 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:56.383755 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:56.383755 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:56.383755 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:56.383755 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:07:56.383755 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:07:56.383755 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:07:56.383755 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:07:56.383755 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:07:56.383755 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:07:56.383755 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:07:56.383755 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:56.384383 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 258/503] 10.222.0.81 () {56 vars in 1102 bytes} [Tue Jun 7 14:07:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:56.385132 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bfa4e-7adf-46a4-8e1b-502d330ea437 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:56.385412 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bfa4e-7adf-46a4-8e1b-502d330ea437 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:56.385684 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bfa4e-7adf-46a4-8e1b-502d330ea437 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:56.387112 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 259/504] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:07:56.387680 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741bfa4e-7adf-46a4-8e1b-502d330ea437 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:56.414931 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 246/505] 10.222.0.81 () {56 vars in 1133 bytes} [Tue Jun 7 14:07:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:56.414931 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ccfa0-039d-4fcc-944b-9db62329fdd1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:56.471049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:56.488812 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741bfa4e-7adf-46a4-8e1b-502d330ea437 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:56.506585 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:56.512996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ccfa0-039d-4fcc-944b-9db62329fdd1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:56.524667 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054ede8-0b0b-4f7d-8512-7b4d2254041b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:56.525327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054ede8-0b0b-4f7d-8512-7b4d2254041b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:56.525610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054ede8-0b0b-4f7d-8512-7b4d2254041b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:56.525894 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054ede8-0b0b-4f7d-8512-7b4d2254041b None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:56.526613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0054ede8-0b0b-4f7d-8512-7b4d2254041b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:56.526916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0054ede8-0b0b-4f7d-8512-7b4d2254041b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:56.528316 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0054ede8-0b0b-4f7d-8512-7b4d2254041b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:56.540256 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 260/506] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:07:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:56.541051 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054ede8-0b0b-4f7d-8512-7b4d2254041b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:56.552176 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d24fd9-0a35-4313-97de-4baed3d99391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:56.552916 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d24fd9-0a35-4313-97de-4baed3d99391 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:56.553106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d24fd9-0a35-4313-97de-4baed3d99391 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:56.553380 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d24fd9-0a35-4313-97de-4baed3d99391 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:56.622864 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d24fd9-0a35-4313-97de-4baed3d99391 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:56.624265 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d24fd9-0a35-4313-97de-4baed3d99391 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:56.640743 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054ede8-0b0b-4f7d-8512-7b4d2254041b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:56.673502 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e1e58-e54d-4f65-874e-2f381e20bbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:56.674161 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e1e58-e54d-4f65-874e-2f381e20bbf0 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:56.674444 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e1e58-e54d-4f65-874e-2f381e20bbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:56.674720 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e1e58-e54d-4f65-874e-2f381e20bbf0 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:56.675545 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17e1e58-e54d-4f65-874e-2f381e20bbf0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:56.675852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17e1e58-e54d-4f65-874e-2f381e20bbf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:56.677202 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17e1e58-e54d-4f65-874e-2f381e20bbf0 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:56.680632 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:56.685735 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 261/507] 10.222.0.81 () {56 vars in 1103 bytes} [Tue Jun 7 14:07:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:56.686921 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e71acc-fd87-4537-9a16-77948b21f4fe None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:56.687210 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e71acc-fd87-4537-9a16-77948b21f4fe None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:56.687516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e71acc-fd87-4537-9a16-77948b21f4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:56.703070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-72e71acc-fd87-4537-9a16-77948b21f4fe None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:07:56.704280 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-72e71acc-fd87-4537-9a16-77948b21f4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:07:56.722139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 247/508] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:07:56.723207 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d24fd9-0a35-4313-97de-4baed3d99391 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:56.735346 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:56.787241 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 262/509] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:07:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:07:56.787241 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e71acc-fd87-4537-9a16-77948b21f4fe None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:56.823277 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d24fd9-0a35-4313-97de-4baed3d99391 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:56.857211 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5ed23-138b-4988-bf78-9903fc8858b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:56.857857 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5ed23-138b-4988-bf78-9903fc8858b3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:56.858140 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5ed23-138b-4988-bf78-9903fc8858b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:56.858412 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5ed23-138b-4988-bf78-9903fc8858b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:56.871381 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e71acc-fd87-4537-9a16-77948b21f4fe None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:56.887558 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:56.906532 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a75bf-0b0b-4fb4-a907-2a562d9f9d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:56.907351 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a75bf-0b0b-4fb4-a907-2a562d9f9d85 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:56.907637 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a75bf-0b0b-4fb4-a907-2a562d9f9d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:56.907908 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a75bf-0b0b-4fb4-a907-2a562d9f9d85 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:56.915141 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a75bf-0b0b-4fb4-a907-2a562d9f9d85 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:56.916610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a75bf-0b0b-4fb4-a907-2a562d9f9d85 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:56.927415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a5ed23-138b-4988-bf78-9903fc8858b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:56.929345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a5ed23-138b-4988-bf78-9903fc8858b3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:56.931296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-1e3a75bf-0b0b-4fb4-a907-2a562d9f9d85 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 14:07:56.931296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:56.931296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:56.931296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:56.931296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:56.931296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:56.931296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:56.931296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:56.931296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:56.931296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:56.931296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:56.931296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:56.931296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:07:56.931296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:07:56.931296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:07:56.931296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:07:56.932192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:56.932192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:56.932192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:07:56.932192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:07:56.932192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:07:56.932192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:07:56.932192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:56.932192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:56.932192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:56.932192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:56.932192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:56.932192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:56.932192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:56.932192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:56.932192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:56.932192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:56.932192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:56.932192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:56.933253 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:56.933253 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:56.933253 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:07:56.933253 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:07:56.933253 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:07:56.933253 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:07:56.933253 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:07:56.933253 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:07:56.933253 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 14:07:56.933253 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:56.933253 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 263/510] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:07:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:56.933253 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a75bf-0b0b-4fb4-a907-2a562d9f9d85 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:56.962544 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:57.032341 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 248/511] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:07:57.033185 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5ed23-138b-4988-bf78-9903fc8858b3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:57.041043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a75bf-0b0b-4fb4-a907-2a562d9f9d85 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:57.073334 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58571d4d-c080-4561-bc50-0ad89b436c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:57.074119 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58571d4d-c080-4561-bc50-0ad89b436c94 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:57.074403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58571d4d-c080-4561-bc50-0ad89b436c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:57.074677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58571d4d-c080-4561-bc50-0ad89b436c94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:57.084908 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58571d4d-c080-4561-bc50-0ad89b436c94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:57.086921 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58571d4d-c080-4561-bc50-0ad89b436c94 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:57.090266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:57.104368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-58571d4d-c080-4561-bc50-0ad89b436c94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:07:57.104368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:57.104368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:57.104368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:57.104368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:57.104368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:57.104368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:57.104368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:57.104368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:57.104368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:57.104368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:57.104368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:57.104368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:07:57.104368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:07:57.104368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:07:57.105305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:07:57.105305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:57.105305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:57.105305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:57.105305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:57.105305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:57.105305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:57.105305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:57.105305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:57.105305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:57.105305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:57.105305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:57.105305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:57.105305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:57.105305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:57.105305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:57.105305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:57.106158 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:07:57.106158 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:07:57.106158 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:07:57.106158 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:57.106158 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 264/512] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:07:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:57.106430 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58571d4d-c080-4561-bc50-0ad89b436c94 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:57.132538 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5ed23-138b-4988-bf78-9903fc8858b3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:57.168684 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d70065-abd0-4162-aff1-a9436877f815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:57.169491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d70065-abd0-4162-aff1-a9436877f815 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:57.169777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d70065-abd0-4162-aff1-a9436877f815 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:57.170055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d70065-abd0-4162-aff1-a9436877f815 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:57.180277 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d70065-abd0-4162-aff1-a9436877f815 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:57.181716 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d70065-abd0-4162-aff1-a9436877f815 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:57.192777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:57.200121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-d0d70065-abd0-4162-aff1-a9436877f815 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:07:57.200121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:57.200121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:57.200121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:57.200121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:57.200121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:57.200121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:57.200121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:57.200121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:57.200121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:57.200121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:57.200121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:57.200121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:07:57.200121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:07:57.200121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:07:57.201051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:07:57.201051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:57.201051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:57.201051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:57.201051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:57.201051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:57.201051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:57.201051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:57.201051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:57.201051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:57.201051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:57.201051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:57.201051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:57.201051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:57.201051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:57.201051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:57.201051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:57.201926 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:07:57.201926 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:07:57.201926 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:07:57.201926 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:57.201926 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 249/513] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:57.202196 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d70065-abd0-4162-aff1-a9436877f815 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:57.214830 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58571d4d-c080-4561-bc50-0ad89b436c94 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:57.248677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cda297-42f0-4157-b852-a143088b29ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:57.249493 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cda297-42f0-4157-b852-a143088b29ea None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:57.249776 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cda297-42f0-4157-b852-a143088b29ea None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:57.250053 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cda297-42f0-4157-b852-a143088b29ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:57.260575 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cda297-42f0-4157-b852-a143088b29ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:57.262036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cda297-42f0-4157-b852-a143088b29ea None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:57.279726 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-57cda297-42f0-4157-b852-a143088b29ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:07:57.279726 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:57.279726 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:57.279726 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:57.279726 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:57.279726 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:57.279726 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:57.279726 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:57.279726 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:57.279726 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:57.279726 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:57.279726 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:57.279726 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:07:57.279726 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:07:57.279726 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:07:57.280759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:07:57.280759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:57.280759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:57.280759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:57.280759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:57.280759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:57.280759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:57.280759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:57.280759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:57.280759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:57.280759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:57.280759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:57.280759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:57.280759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:57.280759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:57.280759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:57.280759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:57.281612 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:07:57.281612 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:07:57.281612 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:07:57.281612 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:57.281612 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 265/514] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:57.281886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cda297-42f0-4157-b852-a143088b29ea None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:57.305422 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d70065-abd0-4162-aff1-a9436877f815 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:57.318527 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:57.340288 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24006038-6cab-483b-b9b7-335572b679bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:57.340962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24006038-6cab-483b-b9b7-335572b679bd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:57.341244 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24006038-6cab-483b-b9b7-335572b679bd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:57.341524 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24006038-6cab-483b-b9b7-335572b679bd None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:57.342354 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24006038-6cab-483b-b9b7-335572b679bd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:57.342654 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24006038-6cab-483b-b9b7-335572b679bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:57.344088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24006038-6cab-483b-b9b7-335572b679bd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:57.361510 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 250/515] 10.222.0.81 () {56 vars in 1137 bytes} [Tue Jun 7 14:07:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:57.362246 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24006038-6cab-483b-b9b7-335572b679bd None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:57.381180 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cda297-42f0-4157-b852-a143088b29ea None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:57.409457 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:57.415374 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3627c-d216-4750-b2a2-232d6dea10c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:57.416211 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3627c-d216-4750-b2a2-232d6dea10c6 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:57.416441 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3627c-d216-4750-b2a2-232d6dea10c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:57.416714 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3627c-d216-4750-b2a2-232d6dea10c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:57.426943 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb3627c-d216-4750-b2a2-232d6dea10c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:57.428407 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb3627c-d216-4750-b2a2-232d6dea10c6 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:57.446168 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-fdb3627c-d216-4750-b2a2-232d6dea10c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:07:57.446168 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:57.446168 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:57.446168 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:57.446168 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:57.446168 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:57.446168 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:57.446168 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:57.446168 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:57.446168 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:57.446168 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:57.446168 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:57.446168 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:07:57.446168 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:07:57.446168 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:07:57.447025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:07:57.447025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:57.447025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:57.447025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:57.447025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:57.447025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:57.447025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:57.447025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:57.447025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:57.447025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:57.447025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:57.447025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:57.447025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:57.447025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:57.447025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:57.447025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:57.447025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:57.448119 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:07:57.448119 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:07:57.448119 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:07:57.448119 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:57.448119 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 266/516] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:57.448119 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3627c-d216-4750-b2a2-232d6dea10c6 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:57.463709 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24006038-6cab-483b-b9b7-335572b679bd None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:57.498196 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8599a1-5956-4f39-a48e-8ef883a4592c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:57.498857 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8599a1-5956-4f39-a48e-8ef883a4592c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:57.499137 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8599a1-5956-4f39-a48e-8ef883a4592c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:57.499438 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8599a1-5956-4f39-a48e-8ef883a4592c None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:57.500155 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8599a1-5956-4f39-a48e-8ef883a4592c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:57.500458 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8599a1-5956-4f39-a48e-8ef883a4592c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:57.501809 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8599a1-5956-4f39-a48e-8ef883a4592c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:57.512874 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 251/517] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:57.513660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8599a1-5956-4f39-a48e-8ef883a4592c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:57.542603 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:57.547673 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3627c-d216-4750-b2a2-232d6dea10c6 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:57.581785 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f9678-bfe4-4ba7-a2bf-f406c3898f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:57.582449 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f9678-bfe4-4ba7-a2bf-f406c3898f53 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:57.582736 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f9678-bfe4-4ba7-a2bf-f406c3898f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:57.583014 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f9678-bfe4-4ba7-a2bf-f406c3898f53 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:57.583772 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72f9678-bfe4-4ba7-a2bf-f406c3898f53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:57.584082 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72f9678-bfe4-4ba7-a2bf-f406c3898f53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:57.587424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72f9678-bfe4-4ba7-a2bf-f406c3898f53 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:57.599385 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 267/518] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:57.600175 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f9678-bfe4-4ba7-a2bf-f406c3898f53 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:57.615838 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8599a1-5956-4f39-a48e-8ef883a4592c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:57.645413 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:57.649648 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897a02d-e6d0-4970-9360-f3c2246f84a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:57.650315 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897a02d-e6d0-4970-9360-f3c2246f84a9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:57.650598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897a02d-e6d0-4970-9360-f3c2246f84a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:57.650872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897a02d-e6d0-4970-9360-f3c2246f84a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:57.651625 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5897a02d-e6d0-4970-9360-f3c2246f84a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:57.651932 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5897a02d-e6d0-4970-9360-f3c2246f84a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:57.653296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5897a02d-e6d0-4970-9360-f3c2246f84a9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:57.665136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 252/519] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:57.665923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897a02d-e6d0-4970-9360-f3c2246f84a9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:57.699780 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f9678-bfe4-4ba7-a2bf-f406c3898f53 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:57.731846 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91de47-dc3b-4774-97d1-9fc3389d40a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:57.732491 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91de47-dc3b-4774-97d1-9fc3389d40a0 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:57.732775 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91de47-dc3b-4774-97d1-9fc3389d40a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:57.733050 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91de47-dc3b-4774-97d1-9fc3389d40a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:57.756537 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:57.767503 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897a02d-e6d0-4970-9360-f3c2246f84a9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:57.800128 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3813e5b-b7e0-4e0d-bcd6-ecc4240af093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:57.800788 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3813e5b-b7e0-4e0d-bcd6-ecc4240af093 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:57.801065 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3813e5b-b7e0-4e0d-bcd6-ecc4240af093 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:57.801332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3813e5b-b7e0-4e0d-bcd6-ecc4240af093 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:57.801603 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae91de47-dc3b-4774-97d1-9fc3389d40a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:57.802043 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3813e5b-b7e0-4e0d-bcd6-ecc4240af093 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:57.802342 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3813e5b-b7e0-4e0d-bcd6-ecc4240af093 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:57.803070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae91de47-dc3b-4774-97d1-9fc3389d40a0 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:57.803710 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3813e5b-b7e0-4e0d-bcd6-ecc4240af093 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:57.816564 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 253/520] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:57.817366 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3813e5b-b7e0-4e0d-bcd6-ecc4240af093 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:57.851799 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:57.909059 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 268/521] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:07:57.909893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91de47-dc3b-4774-97d1-9fc3389d40a0 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:57.919040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3813e5b-b7e0-4e0d-bcd6-ecc4240af093 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:57.952643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461bfa7-c8e5-4b4a-819b-b7f43e76a3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:57.953363 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461bfa7-c8e5-4b4a-819b-b7f43e76a3b4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:57.953653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461bfa7-c8e5-4b4a-819b-b7f43e76a3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:57.953931 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461bfa7-c8e5-4b4a-819b-b7f43e76a3b4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:57.961846 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1461bfa7-c8e5-4b4a-819b-b7f43e76a3b4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:57.963295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1461bfa7-c8e5-4b4a-819b-b7f43e76a3b4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:57.977046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-1461bfa7-c8e5-4b4a-819b-b7f43e76a3b4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 14:07:57.977046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:57.977046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:57.977046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:57.977046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:57.977046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:57.977046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:57.977046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:57.977046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:57.977046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:57.977046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:57.977046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:57.977046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:07:57.977046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:07:57.977046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:07:57.977046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:07:57.978024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:57.978024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:57.978024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:07:57.978024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:07:57.978024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:07:57.978024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:07:57.978024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:57.978024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:57.978024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:57.978024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:57.978024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:57.978024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:57.978024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:57.978024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:57.978024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:57.978024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:57.978024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:57.978024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:57.979042 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:57.979042 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:57.979042 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:07:57.979042 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:07:57.979042 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:07:57.979042 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:07:57.979042 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:07:57.979042 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:07:57.979042 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 14:07:57.979042 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:57.979042 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 254/522] 10.222.0.81 () {56 vars in 1102 bytes} [Tue Jun 7 14:07:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:57.980130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461bfa7-c8e5-4b4a-819b-b7f43e76a3b4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:57.992676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:58.007811 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91de47-dc3b-4774-97d1-9fc3389d40a0 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:58.042717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3f9d4-81b8-49cb-990a-6acc3793242e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:58.043535 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3f9d4-81b8-49cb-990a-6acc3793242e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:58.043816 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3f9d4-81b8-49cb-990a-6acc3793242e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:58.044086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3f9d4-81b8-49cb-990a-6acc3793242e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:58.050615 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3f9d4-81b8-49cb-990a-6acc3793242e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:58.052074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3f9d4-81b8-49cb-990a-6acc3793242e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:58.066205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-5de3f9d4-81b8-49cb-990a-6acc3793242e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:07:58.066205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:58.066205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:58.066205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:58.066205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:58.066205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:58.066205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:58.066205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:58.066205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:58.066205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:58.066205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:58.066205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:58.066205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:07:58.066205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:07:58.066205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:07:58.067250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:07:58.067250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:58.067250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:58.067250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:58.067250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:58.067250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:58.067250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:58.067250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:58.067250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:58.067250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:58.067250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:58.067250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:58.067250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:58.067250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:58.067250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:58.067250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:58.067250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:58.068218 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:07:58.068218 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:07:58.068218 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:07:58.068218 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:07:58.068218 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:07:58.068218 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:07:58.068218 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:07:58.068218 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:58.068218 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 269/523] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:07:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:58.068218 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3f9d4-81b8-49cb-990a-6acc3793242e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:58.073324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:58.084852 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461bfa7-c8e5-4b4a-819b-b7f43e76a3b4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:58.118972 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67f47f-01f0-4fb6-a72b-2e06869d679b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:58.119674 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67f47f-01f0-4fb6-a72b-2e06869d679b None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:58.119963 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67f47f-01f0-4fb6-a72b-2e06869d679b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:58.120245 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67f47f-01f0-4fb6-a72b-2e06869d679b None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:58.121218 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67f47f-01f0-4fb6-a72b-2e06869d679b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:58.122652 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67f47f-01f0-4fb6-a72b-2e06869d679b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:58.142143 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 255/524] 10.222.0.81 () {60 vars in 1132 bytes} [Tue Jun 7 14:07:57 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:07:58.143039 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67f47f-01f0-4fb6-a72b-2e06869d679b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:58.202352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3f9d4-81b8-49cb-990a-6acc3793242e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:58.223418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:58.241990 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87412421-f49f-41f3-9b03-516e22a99a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:58.242821 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87412421-f49f-41f3-9b03-516e22a99a0c None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:58.243107 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87412421-f49f-41f3-9b03-516e22a99a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:58.243415 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87412421-f49f-41f3-9b03-516e22a99a0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:58.251699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87412421-f49f-41f3-9b03-516e22a99a0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:58.253212 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87412421-f49f-41f3-9b03-516e22a99a0c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:58.268767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-87412421-f49f-41f3-9b03-516e22a99a0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:07:58.268767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:58.268767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:58.268767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:58.268767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:58.268767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:58.268767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:58.268767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:58.268767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:58.268767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:58.268767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:58.268767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:58.268767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:07:58.268767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:07:58.268767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:07:58.269725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:07:58.269725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:58.269725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:58.269725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:58.269725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:58.269725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:58.269725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:58.269725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:58.269725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:58.269725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:58.269725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:58.269725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:58.269725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:58.269725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:58.269725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:58.269725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:58.269725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:58.270599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:07:58.270599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:07:58.270599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:07:58.270599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:07:58.270599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:07:58.270599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:07:58.270599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:07:58.270599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:58.271088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67f47f-01f0-4fb6-a72b-2e06869d679b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:58.271165 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 270/525] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:07:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:58.271165 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87412421-f49f-41f3-9b03-516e22a99a0c None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:58.313288 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316cdfeb-ce2f-4aa9-82f8-649128e75a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:58.314108 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316cdfeb-ce2f-4aa9-82f8-649128e75a7b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:58.314395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316cdfeb-ce2f-4aa9-82f8-649128e75a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:58.314668 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316cdfeb-ce2f-4aa9-82f8-649128e75a7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:58.327263 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316cdfeb-ce2f-4aa9-82f8-649128e75a7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:58.328736 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316cdfeb-ce2f-4aa9-82f8-649128e75a7b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:58.348088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-316cdfeb-ce2f-4aa9-82f8-649128e75a7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:07:58.348088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:58.348088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:58.348088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:58.348088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:58.348088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:58.348088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:58.348088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:58.348088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:58.348088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:58.348088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:58.348088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:58.348088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:07:58.348088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:07:58.348088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:07:58.348999 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:07:58.348999 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:58.348999 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:58.348999 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:58.348999 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:58.348999 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:58.348999 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:58.348999 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:58.348999 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:58.348999 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:58.348999 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:58.348999 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:58.348999 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:58.348999 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:58.348999 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:58.348999 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:58.348999 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:58.349866 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:07:58.349866 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:07:58.349866 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:07:58.349866 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:58.349866 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 256/526] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:07:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:58.350144 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316cdfeb-ce2f-4aa9-82f8-649128e75a7b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:58.356678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:58.398629 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87412421-f49f-41f3-9b03-516e22a99a0c None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:58.439602 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2fbb98-3a18-47c6-bc9d-911a0a8954d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:58.440275 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fbb98-3a18-47c6-bc9d-911a0a8954d0 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:58.440608 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fbb98-3a18-47c6-bc9d-911a0a8954d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:58.441238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fbb98-3a18-47c6-bc9d-911a0a8954d0 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:58.442423 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2fbb98-3a18-47c6-bc9d-911a0a8954d0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:58.442742 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2fbb98-3a18-47c6-bc9d-911a0a8954d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:58.444351 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2fbb98-3a18-47c6-bc9d-911a0a8954d0 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:58.462528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 271/527] 10.222.0.81 () {56 vars in 1139 bytes} [Tue Jun 7 14:07:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:58.463376 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2fbb98-3a18-47c6-bc9d-911a0a8954d0 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:58.469424 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316cdfeb-ce2f-4aa9-82f8-649128e75a7b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:58.488410 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:58.508873 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783f70d-ebea-4223-9890-5522de414aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:58.509864 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783f70d-ebea-4223-9890-5522de414aff None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:58.510348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783f70d-ebea-4223-9890-5522de414aff None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:58.510806 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783f70d-ebea-4223-9890-5522de414aff None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:58.511929 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c783f70d-ebea-4223-9890-5522de414aff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:58.512547 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c783f70d-ebea-4223-9890-5522de414aff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:58.514003 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c783f70d-ebea-4223-9890-5522de414aff None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:58.534989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 257/528] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:07:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:58.535945 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783f70d-ebea-4223-9890-5522de414aff None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:58.574538 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2fbb98-3a18-47c6-bc9d-911a0a8954d0 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:58.599945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:58.613693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429773f0-58d5-4cee-b26a-14ae4647997d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:58.614362 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429773f0-58d5-4cee-b26a-14ae4647997d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:58.614646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429773f0-58d5-4cee-b26a-14ae4647997d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:58.614924 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429773f0-58d5-4cee-b26a-14ae4647997d None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:58.615777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429773f0-58d5-4cee-b26a-14ae4647997d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:58.616079 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429773f0-58d5-4cee-b26a-14ae4647997d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:58.618103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429773f0-58d5-4cee-b26a-14ae4647997d None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:58.642008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 272/529] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:07:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:58.642008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429773f0-58d5-4cee-b26a-14ae4647997d None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:58.660372 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783f70d-ebea-4223-9890-5522de414aff None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:58.702842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768ccfd-3032-4ec6-9be9-acfdf22f0ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:58.703901 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768ccfd-3032-4ec6-9be9-acfdf22f0ada None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:58.704383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768ccfd-3032-4ec6-9be9-acfdf22f0ada None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:58.704828 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768ccfd-3032-4ec6-9be9-acfdf22f0ada None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:58.705721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b768ccfd-3032-4ec6-9be9-acfdf22f0ada None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:58.706193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b768ccfd-3032-4ec6-9be9-acfdf22f0ada None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:58.707779 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b768ccfd-3032-4ec6-9be9-acfdf22f0ada None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:58.721507 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 258/530] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:07:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:58.722898 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768ccfd-3032-4ec6-9be9-acfdf22f0ada None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:58.747174 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429773f0-58d5-4cee-b26a-14ae4647997d None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:58.767490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:58.785118 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f538c7-5e17-4da0-88c1-aafd9f058aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:58.785773 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f538c7-5e17-4da0-88c1-aafd9f058aac None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:58.786064 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f538c7-5e17-4da0-88c1-aafd9f058aac None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:58.786344 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f538c7-5e17-4da0-88c1-aafd9f058aac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:58.832265 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768ccfd-3032-4ec6-9be9-acfdf22f0ada None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:58.842648 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:58.864778 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f538c7-5e17-4da0-88c1-aafd9f058aac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:58.866748 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f538c7-5e17-4da0-88c1-aafd9f058aac None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:58.869171 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5fb62-93ab-4a07-a037-7d90b33ce87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:58.869832 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5fb62-93ab-4a07-a037-7d90b33ce87c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:58.870120 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5fb62-93ab-4a07-a037-7d90b33ce87c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:58.870400 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5fb62-93ab-4a07-a037-7d90b33ce87c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:58.871549 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5fb62-93ab-4a07-a037-7d90b33ce87c None admin] RBAC: Adding query filter params (role.id=a72e5464b7b746978805326c224753bc, scope.project.id=f67bad575f8b4a47bee7c314497cb2fe, user.id=b6180393da0f495db964f0b2588d8b90) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:58.871864 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5fb62-93ab-4a07-a037-7d90b33ce87c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:58.873296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5fb62-93ab-4a07-a037-7d90b33ce87c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:58.883150 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 259/531] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:07:58 2022] GET /identity/v3/role_assignments?user.id=b6180393da0f495db964f0b2588d8b90&scope.project.id=f67bad575f8b4a47bee7c314497cb2fe&role.id=a72e5464b7b746978805326c224753bc => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:58.883150 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5fb62-93ab-4a07-a037-7d90b33ce87c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:59.019482 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 273/532] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:07:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:07:59.020731 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f538c7-5e17-4da0-88c1-aafd9f058aac None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:59.024177 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5fb62-93ab-4a07-a037-7d90b33ce87c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:59.026333 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:59.066227 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10cf62-f791-4003-bb5b-5b5003280a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:59.067243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10cf62-f791-4003-bb5b-5b5003280a5e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:59.067709 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10cf62-f791-4003-bb5b-5b5003280a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:59.068157 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10cf62-f791-4003-bb5b-5b5003280a5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:59.069178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e10cf62-f791-4003-bb5b-5b5003280a5e None admin] RBAC: Adding query filter params (role.id=a72e5464b7b746978805326c224753bc, scope.project.id=f67bad575f8b4a47bee7c314497cb2fe, user.id=2ae726fbc9cf40e68c7587552a118a7a) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:59.069865 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e10cf62-f791-4003-bb5b-5b5003280a5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:59.071631 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e10cf62-f791-4003-bb5b-5b5003280a5e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:59.078825 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 260/533] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:07:58 2022] GET /identity/v3/role_assignments?user.id=2ae726fbc9cf40e68c7587552a118a7a&scope.project.id=f67bad575f8b4a47bee7c314497cb2fe&role.id=a72e5464b7b746978805326c224753bc => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:59.080090 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10cf62-f791-4003-bb5b-5b5003280a5e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:59.143887 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:59.177176 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f538c7-5e17-4da0-88c1-aafd9f058aac None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:59.224847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10cf62-f791-4003-bb5b-5b5003280a5e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:59.240035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc292ea-6ac8-493d-b9bf-07f006c4dbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:59.240871 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc292ea-6ac8-493d-b9bf-07f006c4dbb1 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:59.241155 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc292ea-6ac8-493d-b9bf-07f006c4dbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:59.241677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc292ea-6ac8-493d-b9bf-07f006c4dbb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:59.249891 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc292ea-6ac8-493d-b9bf-07f006c4dbb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:59.258777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc292ea-6ac8-493d-b9bf-07f006c4dbb1 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:59.274682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-bbc292ea-6ac8-493d-b9bf-07f006c4dbb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:07:59.274682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:59.274682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:59.274682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:59.274682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:59.274682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:59.274682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:59.274682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:59.274682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:59.274682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:59.274682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:59.274682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:59.274682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:07:59.274682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:07:59.274682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:07:59.275614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:07:59.275614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:59.275614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:59.275614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:59.275614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:59.275614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:59.275614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:59.275614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:59.275614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:59.275614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:59.275614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:59.275614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:59.275614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:59.275614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:59.275614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:59.275614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:59.275614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:59.276981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:07:59.276981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:07:59.276981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:07:59.276981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:07:59.276981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:07:59.276981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:07:59.276981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:07:59.276981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:07:59.276981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 274/534] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:07:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:59.278875 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf29f8-7844-4c78-85e3-a7ba81d76686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:59.279420 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf29f8-7844-4c78-85e3-a7ba81d76686 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:59.279707 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf29f8-7844-4c78-85e3-a7ba81d76686 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:59.279983 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf29f8-7844-4c78-85e3-a7ba81d76686 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:59.288910 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc292ea-6ac8-493d-b9bf-07f006c4dbb1 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:59.295828 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf29f8-7844-4c78-85e3-a7ba81d76686 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:59.297320 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf29f8-7844-4c78-85e3-a7ba81d76686 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:59.332283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-67bf29f8-7844-4c78-85e3-a7ba81d76686 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:07:59.332283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:59.332283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:59.332283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:59.332283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:59.332283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:59.332283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:59.332283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:59.332283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:59.332283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:59.332283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:59.332283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:59.332283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:07:59.332283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:07:59.332283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:07:59.333187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:07:59.333187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:59.333187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:59.333187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:59.333187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:59.333187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:59.333187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:59.333187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:59.333187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:59.333187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:59.333187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:59.333187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:59.333187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:59.333187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:59.333187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:59.333187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:59.333187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:59.334899 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:07:59.334899 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:07:59.334899 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:07:59.334899 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:59.334899 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 261/535] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:07:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:59.339872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf29f8-7844-4c78-85e3-a7ba81d76686 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:59.379612 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:59.405045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc292ea-6ac8-493d-b9bf-07f006c4dbb1 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:59.447024 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b4f7b-510d-4b10-8be5-36cb7925af56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:59.447735 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b4f7b-510d-4b10-8be5-36cb7925af56 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:59.448023 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b4f7b-510d-4b10-8be5-36cb7925af56 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:59.448299 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b4f7b-510d-4b10-8be5-36cb7925af56 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:59.449094 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b4f7b-510d-4b10-8be5-36cb7925af56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:59.449405 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b4f7b-510d-4b10-8be5-36cb7925af56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:59.450860 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b4f7b-510d-4b10-8be5-36cb7925af56 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:59.478377 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 275/536] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:07:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:07:59.484463 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b4f7b-510d-4b10-8be5-36cb7925af56 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:59.489952 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:59.527036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf29f8-7844-4c78-85e3-a7ba81d76686 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:59.595943 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616dccd-14ef-4c52-9e42-30fa19a1aab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:59.596630 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616dccd-14ef-4c52-9e42-30fa19a1aab6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:59.596916 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616dccd-14ef-4c52-9e42-30fa19a1aab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:59.597192 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616dccd-14ef-4c52-9e42-30fa19a1aab6 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:59.597926 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e616dccd-14ef-4c52-9e42-30fa19a1aab6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:59.598235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e616dccd-14ef-4c52-9e42-30fa19a1aab6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:59.599641 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e616dccd-14ef-4c52-9e42-30fa19a1aab6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:59.614240 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b4f7b-510d-4b10-8be5-36cb7925af56 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:59.625713 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 262/537] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:59.632531 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616dccd-14ef-4c52-9e42-30fa19a1aab6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:07:59.655324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7837dc25-58a5-4281-ae38-a0213c8feebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:59.655992 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837dc25-58a5-4281-ae38-a0213c8feebd None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:59.656281 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837dc25-58a5-4281-ae38-a0213c8feebd None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:59.656565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837dc25-58a5-4281-ae38-a0213c8feebd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:59.657405 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7837dc25-58a5-4281-ae38-a0213c8feebd None admin] RBAC: Adding query filter params (role.id=a72e5464b7b746978805326c224753bc, scope.project.id=f67bad575f8b4a47bee7c314497cb2fe, user.id=38b1ff21b8e5445b878740a327e4442f) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:07:59.657721 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7837dc25-58a5-4281-ae38-a0213c8feebd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:59.659152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7837dc25-58a5-4281-ae38-a0213c8feebd None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:59.665971 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 276/538] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:07:59 2022] GET /identity/v3/role_assignments?user.id=38b1ff21b8e5445b878740a327e4442f&scope.project.id=f67bad575f8b4a47bee7c314497cb2fe&role.id=a72e5464b7b746978805326c224753bc => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:07:59.771159 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:07:59.800570 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616dccd-14ef-4c52-9e42-30fa19a1aab6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:07:59.854983 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22acfc-6c8e-48c7-8550-85cdf41351d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:07:59.855835 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22acfc-6c8e-48c7-8550-85cdf41351d9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:07:59.856116 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22acfc-6c8e-48c7-8550-85cdf41351d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:07:59.856390 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22acfc-6c8e-48c7-8550-85cdf41351d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:07:59.869837 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b22acfc-6c8e-48c7-8550-85cdf41351d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:07:59.871578 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b22acfc-6c8e-48c7-8550-85cdf41351d9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:07:59.890622 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-6b22acfc-6c8e-48c7-8550-85cdf41351d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:07:59.890622 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:07:59.890622 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:07:59.890622 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:07:59.890622 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:07:59.890622 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:07:59.890622 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:07:59.890622 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:07:59.890622 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:07:59.890622 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:07:59.890622 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:07:59.890622 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:07:59.890622 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:07:59.890622 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:07:59.890622 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:07:59.891589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:07:59.891589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:07:59.891589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:07:59.891589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:07:59.891589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:07:59.891589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:07:59.891589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:07:59.891589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:07:59.891589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:07:59.891589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:07:59.891589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:07:59.891589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:07:59.891589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:07:59.891589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:07:59.891589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:07:59.891589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:07:59.891589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:07:59.893201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:07:59.893201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:07:59.893201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:07:59.893201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:07:59.893201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:07:59.893201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:07:59.893201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:07:59.893201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:07:59.893201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 263/539] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:07:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:07:59.900595 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7837dc25-58a5-4281-ae38-a0213c8feebd None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:00.030172 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:00.075912 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7837dc25-58a5-4281-ae38-a0213c8feebd None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:00.114644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305a04c-335e-4b07-bc77-63e62af42b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:00.115351 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305a04c-335e-4b07-bc77-63e62af42b72 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:00.115639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305a04c-335e-4b07-bc77-63e62af42b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:00.115920 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305a04c-335e-4b07-bc77-63e62af42b72 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:00.116759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305a04c-335e-4b07-bc77-63e62af42b72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:00.117158 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305a04c-335e-4b07-bc77-63e62af42b72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:00.118597 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305a04c-335e-4b07-bc77-63e62af42b72 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:00.132533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 277/540] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:07:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:00.138627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22acfc-6c8e-48c7-8550-85cdf41351d9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:00.272266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22acfc-6c8e-48c7-8550-85cdf41351d9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:00.319535 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ddcf9d-0fe6-4c43-909d-87fb2c47207c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:00.320819 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddcf9d-0fe6-4c43-909d-87fb2c47207c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:00.321320 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddcf9d-0fe6-4c43-909d-87fb2c47207c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:00.321795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddcf9d-0fe6-4c43-909d-87fb2c47207c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:00.322825 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ddcf9d-0fe6-4c43-909d-87fb2c47207c None admin] RBAC: Adding query filter params (role.id=a72e5464b7b746978805326c224753bc, scope.project.id=f67bad575f8b4a47bee7c314497cb2fe, user.id=a6a08eaecbcd41228571739d8387882a) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:00.323373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ddcf9d-0fe6-4c43-909d-87fb2c47207c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:00.325017 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ddcf9d-0fe6-4c43-909d-87fb2c47207c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:00.331664 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 264/541] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:08:00 2022] GET /identity/v3/role_assignments?user.id=a6a08eaecbcd41228571739d8387882a&scope.project.id=f67bad575f8b4a47bee7c314497cb2fe&role.id=a72e5464b7b746978805326c224753bc => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:00.650605 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8c26d-bd46-4fd5-84fe-cf32f62aa5ef None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:00.651594 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8c26d-bd46-4fd5-84fe-cf32f62aa5ef None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:00.652168 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8c26d-bd46-4fd5-84fe-cf32f62aa5ef None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:00.654034 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 278/542] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:00.659844 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc9780-4d33-41b2-b1f7-55da9cc70a40 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:00.660365 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc9780-4d33-41b2-b1f7-55da9cc70a40 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:00.660842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc9780-4d33-41b2-b1f7-55da9cc70a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:00.667086 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:00.681938 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-95bc9780-4d33-41b2-b1f7-55da9cc70a40 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:00.683469 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-95bc9780-4d33-41b2-b1f7-55da9cc70a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:00.767302 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 265/543] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:08:00.787675 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8c26d-bd46-4fd5-84fe-cf32f62aa5ef None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:00.932587 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8c26d-bd46-4fd5-84fe-cf32f62aa5ef None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:00.965490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:00.980281 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d17fd4-258e-40a0-825d-d9c408fd4d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:00.981083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d17fd4-258e-40a0-825d-d9c408fd4d11 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:00.981369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d17fd4-258e-40a0-825d-d9c408fd4d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:00.981639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d17fd4-258e-40a0-825d-d9c408fd4d11 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:01.004113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d17fd4-258e-40a0-825d-d9c408fd4d11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:08:01.012524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-14d17fd4-258e-40a0-825d-d9c408fd4d11 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:01.012524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:01.012524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:01.012524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:01.012524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:01.012524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:01.012524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:01.012524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:01.012524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:01.012524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:01.012524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:01.012524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:01.012524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:08:01.012524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:08:01.012524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:08:01.012524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:08:01.013420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:08:01.013420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:08:01.013420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:08:01.013420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:08:01.013420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:01.013420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:01.013420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:01.013420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:01.013420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:01.013420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:01.013420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:01.013420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:01.013420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:01.013420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:01.013420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:01.013420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:01.014655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:01.014655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:01.014655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:01.014655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:01.014655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:08:01.014655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:08:01.014655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:08:01.014655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:08:01.014655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:08:01.014655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:08:01.014655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:01.014655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:01.014655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 279/544] 10.222.0.81 () {56 vars in 1102 bytes} [Tue Jun 7 14:08:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:01.021528 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc9780-4d33-41b2-b1f7-55da9cc70a40 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:01.136180 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc9780-4d33-41b2-b1f7-55da9cc70a40 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:01.172940 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8a907-8795-43b3-9e6e-c703beb27f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:01.173623 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8a907-8795-43b3-9e6e-c703beb27f9b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:01.173914 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8a907-8795-43b3-9e6e-c703beb27f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:01.174188 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8a907-8795-43b3-9e6e-c703beb27f9b None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:01.174952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d8a907-8795-43b3-9e6e-c703beb27f9b None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:01.175293 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d8a907-8795-43b3-9e6e-c703beb27f9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:01.176677 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d8a907-8795-43b3-9e6e-c703beb27f9b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:01.179676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:01.186486 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 266/545] 10.222.0.81 () {56 vars in 1103 bytes} [Tue Jun 7 14:08:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:01.191984 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d17fd4-258e-40a0-825d-d9c408fd4d11 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:01.340296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d17fd4-258e-40a0-825d-d9c408fd4d11 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:01.383340 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffe43a-b9a7-4f14-be9f-d32fa96538d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:01.384147 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffe43a-b9a7-4f14-be9f-d32fa96538d7 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:01.384432 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffe43a-b9a7-4f14-be9f-d32fa96538d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:01.384709 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffe43a-b9a7-4f14-be9f-d32fa96538d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:01.402602 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffe43a-b9a7-4f14-be9f-d32fa96538d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:01.404112 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffe43a-b9a7-4f14-be9f-d32fa96538d7 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:01.425716 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-ebffe43a-b9a7-4f14-be9f-d32fa96538d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:01.425716 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:01.425716 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:01.425716 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:01.425716 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:01.425716 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:01.425716 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:01.425716 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:01.425716 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:01.425716 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:01.425716 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:01.425716 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:01.425716 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:01.425716 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:01.425716 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:01.426631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:01.426631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:01.426631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:01.426631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:01.426631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:01.426631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:01.426631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:01.426631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:01.426631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:01.426631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:01.426631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:01.426631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:01.426631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:01.426631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:01.426631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:01.426631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:01.426631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:01.427953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:01.427953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:01.427953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:01.427953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:01.427953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 280/546] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:01.434776 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8a907-8795-43b3-9e6e-c703beb27f9b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:01.619852 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8a907-8795-43b3-9e6e-c703beb27f9b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:01.677148 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205e94b-43af-4c25-bc87-9c14d3a1088e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:01.677845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205e94b-43af-4c25-bc87-9c14d3a1088e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:01.678133 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205e94b-43af-4c25-bc87-9c14d3a1088e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:01.678413 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205e94b-43af-4c25-bc87-9c14d3a1088e None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:01.679749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205e94b-43af-4c25-bc87-9c14d3a1088e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:01.680059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205e94b-43af-4c25-bc87-9c14d3a1088e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:01.681457 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205e94b-43af-4c25-bc87-9c14d3a1088e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:01.707428 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 267/547] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:01.716366 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffe43a-b9a7-4f14-be9f-d32fa96538d7 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:01.777260 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:01.869718 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffe43a-b9a7-4f14-be9f-d32fa96538d7 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:01.910733 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99892c3c-2e63-499d-ac9d-ea8c6b15cb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:01.911479 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99892c3c-2e63-499d-ac9d-ea8c6b15cb32 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:01.911763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99892c3c-2e63-499d-ac9d-ea8c6b15cb32 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:01.912040 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99892c3c-2e63-499d-ac9d-ea8c6b15cb32 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:01.922680 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99892c3c-2e63-499d-ac9d-ea8c6b15cb32 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:01.924165 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99892c3c-2e63-499d-ac9d-ea8c6b15cb32 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:01.941963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-99892c3c-2e63-499d-ac9d-ea8c6b15cb32 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 14:08:01.941963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:01.941963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:01.941963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:01.941963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:01.941963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:01.941963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:01.941963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:01.941963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:01.941963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:01.941963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:01.941963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:01.941963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:08:01.941963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:08:01.941963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:08:01.941963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:08:01.943501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:01.943501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:01.943501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:08:01.943501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:01.943501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:08:01.943501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:01.943501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:01.943501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:01.943501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:01.943501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:01.943501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:01.943501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:01.943501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:01.943501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:01.943501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:01.943501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:01.943501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:01.943501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:01.945207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:01.945207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:01.945207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:08:01.945207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:08:01.945207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:08:01.945207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:08:01.945207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:08:01.945207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:08:01.945207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 14:08:01.945207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:01.948131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 281/548] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:08:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:01.952027 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1612b-bc40-4aa3-8764-411e017ac56b None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:01.952277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99892c3c-2e63-499d-ac9d-ea8c6b15cb32 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:01.952445 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1612b-bc40-4aa3-8764-411e017ac56b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:01.952701 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1612b-bc40-4aa3-8764-411e017ac56b None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:01.954180 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 268/549] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:01.959554 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbb7ce-7e81-4be9-b340-91ae4c46a392 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:01.961024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbb7ce-7e81-4be9-b340-91ae4c46a392 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:01.961611 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbb7ce-7e81-4be9-b340-91ae4c46a392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:01.998710 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2dcbb7ce-7e81-4be9-b340-91ae4c46a392 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:02.000272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2dcbb7ce-7e81-4be9-b340-91ae4c46a392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:02.065662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:02.086961 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:02.090575 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99892c3c-2e63-499d-ac9d-ea8c6b15cb32 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:02.091958 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 269/550] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:08:02.094110 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe15ad-cd91-4e85-b718-8ea92e98d094 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:02.094631 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe15ad-cd91-4e85-b718-8ea92e98d094 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:02.095141 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe15ad-cd91-4e85-b718-8ea92e98d094 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:02.096918 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 270/551] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:02.102349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3afa3a-b6fe-49c4-8d29-78e1139542d5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:02.102633 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3afa3a-b6fe-49c4-8d29-78e1139542d5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:02.102906 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3afa3a-b6fe-49c4-8d29-78e1139542d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:02.133195 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4f3afa3a-b6fe-49c4-8d29-78e1139542d5 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:02.134489 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4f3afa3a-b6fe-49c4-8d29-78e1139542d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:02.146889 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7a8ad-c384-4e2b-aebb-f226170ed966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:02.148504 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7a8ad-c384-4e2b-aebb-f226170ed966 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:02.149014 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7a8ad-c384-4e2b-aebb-f226170ed966 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:02.149508 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7a8ad-c384-4e2b-aebb-f226170ed966 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:02.150565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7a8ad-c384-4e2b-aebb-f226170ed966 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:02.151136 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7a8ad-c384-4e2b-aebb-f226170ed966 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:02.152824 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7a8ad-c384-4e2b-aebb-f226170ed966 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:02.175164 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 282/552] 10.222.0.81 () {56 vars in 1135 bytes} [Tue Jun 7 14:08:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:02.176392 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7a8ad-c384-4e2b-aebb-f226170ed966 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:02.228092 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 271/553] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:08:02.228092 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3afa3a-b6fe-49c4-8d29-78e1139542d5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:02.320167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7a8ad-c384-4e2b-aebb-f226170ed966 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:02.364197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cecf8dc-7644-412b-8114-f8bc32b4f8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:02.365006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecf8dc-7644-412b-8114-f8bc32b4f8eb None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:02.365289 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecf8dc-7644-412b-8114-f8bc32b4f8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:02.365564 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecf8dc-7644-412b-8114-f8bc32b4f8eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:02.367019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3afa3a-b6fe-49c4-8d29-78e1139542d5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:02.382694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cecf8dc-7644-412b-8114-f8bc32b4f8eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:08:02.387840 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:02.392964 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-4cecf8dc-7644-412b-8114-f8bc32b4f8eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:02.392964 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:02.392964 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:02.392964 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:02.392964 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:02.392964 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:02.392964 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:02.392964 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:02.392964 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:02.392964 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:02.392964 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:02.392964 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:02.392964 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:08:02.392964 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:08:02.392964 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:08:02.392964 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:08:02.394035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:08:02.394035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:08:02.394035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:08:02.394035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:08:02.394035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:02.394035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:02.394035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:02.394035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:02.394035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:02.394035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:02.394035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:02.394035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:02.394035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:02.394035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:02.394035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:02.394035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:02.394885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:02.394885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:02.394885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:02.394885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:02.394885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:08:02.394885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:08:02.394885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:08:02.394885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:08:02.394885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:08:02.394885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:08:02.394885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:02.394885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:02.399301 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 283/554] 10.222.0.81 () {56 vars in 1102 bytes} [Tue Jun 7 14:08:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:02.400403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cecf8dc-7644-412b-8114-f8bc32b4f8eb None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:02.418539 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e1aa9-e3b4-4b67-ab21-976029817e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:02.420250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e1aa9-e3b4-4b67-ab21-976029817e39 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:02.420778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e1aa9-e3b4-4b67-ab21-976029817e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:02.421268 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e1aa9-e3b4-4b67-ab21-976029817e39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:02.425820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:02.516315 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e1aa9-e3b4-4b67-ab21-976029817e39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:02.518593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e1aa9-e3b4-4b67-ab21-976029817e39 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:02.540581 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cecf8dc-7644-412b-8114-f8bc32b4f8eb None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:02.583371 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc67af5-e81f-43c4-ab30-4f4bf0337987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:02.584178 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc67af5-e81f-43c4-ab30-4f4bf0337987 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:02.584464 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc67af5-e81f-43c4-ab30-4f4bf0337987 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:02.584742 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc67af5-e81f-43c4-ab30-4f4bf0337987 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:02.595946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc67af5-e81f-43c4-ab30-4f4bf0337987 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:08:02.604197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-9bc67af5-e81f-43c4-ab30-4f4bf0337987 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:02.604197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:02.604197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:02.604197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:02.604197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:02.604197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:02.604197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:02.604197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:02.604197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:02.604197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:02.604197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:02.604197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:02.604197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:08:02.604197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:08:02.604197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:08:02.604197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:08:02.605238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:08:02.605238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:08:02.605238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:08:02.605238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:08:02.605238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:02.605238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:02.605238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:02.605238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:02.605238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:02.605238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:02.605238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:02.605238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:02.605238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:02.605238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:02.605238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:02.605238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:02.606092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:02.606092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:02.606092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:02.606092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:02.606092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:08:02.606092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:08:02.606092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:08:02.606092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:08:02.606092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:08:02.606092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:08:02.606092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:02.606092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:02.606092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 284/555] 10.222.0.81 () {56 vars in 1102 bytes} [Tue Jun 7 14:08:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:02.606884 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc67af5-e81f-43c4-ab30-4f4bf0337987 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:02.646449 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 272/556] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:08:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:08:02.647913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b983e-e39c-41f5-8356-617b7ea8c22a None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:02.648200 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b983e-e39c-41f5-8356-617b7ea8c22a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:02.648472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b983e-e39c-41f5-8356-617b7ea8c22a None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:02.653643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 273/557] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:02.653643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b983e-e39c-41f5-8356-617b7ea8c22a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:02.668087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:02.701586 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:02.726390 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc67af5-e81f-43c4-ab30-4f4bf0337987 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:02.766122 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b983e-e39c-41f5-8356-617b7ea8c22a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:02.767148 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887dde0-c853-4f6e-826b-a6ee87931122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:02.768045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887dde0-c853-4f6e-826b-a6ee87931122 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:02.768501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887dde0-c853-4f6e-826b-a6ee87931122 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:02.768778 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887dde0-c853-4f6e-826b-a6ee87931122 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:02.769665 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887dde0-c853-4f6e-826b-a6ee87931122 None admin] RBAC: Adding query filter params (name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:02.769973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887dde0-c853-4f6e-826b-a6ee87931122 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:02.771379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887dde0-c853-4f6e-826b-a6ee87931122 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:02.781584 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 285/558] 10.222.0.81 () {56 vars in 1103 bytes} [Tue Jun 7 14:08:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:02.786933 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887dde0-c853-4f6e-826b-a6ee87931122 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:02.803063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fc320-fbac-4185-9056-42d6de95a16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:02.803900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fc320-fbac-4185-9056-42d6de95a16d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:02.804304 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fc320-fbac-4185-9056-42d6de95a16d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:02.804586 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fc320-fbac-4185-9056-42d6de95a16d None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:02.805332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fc320-fbac-4185-9056-42d6de95a16d None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:02.805636 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fc320-fbac-4185-9056-42d6de95a16d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:02.806996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fc320-fbac-4185-9056-42d6de95a16d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:02.815515 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 274/559] 10.222.0.81 () {56 vars in 1103 bytes} [Tue Jun 7 14:08:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:02.816718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30421a1-3a65-46b1-bb73-deb93e18f614 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:02.817001 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30421a1-3a65-46b1-bb73-deb93e18f614 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:02.817272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30421a1-3a65-46b1-bb73-deb93e18f614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:02.833566 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f30421a1-3a65-46b1-bb73-deb93e18f614 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:02.834762 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f30421a1-3a65-46b1-bb73-deb93e18f614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:02.904632 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887dde0-c853-4f6e-826b-a6ee87931122 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:02.906670 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 275/560] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:08:02.907833 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30421a1-3a65-46b1-bb73-deb93e18f614 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:02.917126 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:02.946603 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d362c0-3817-4ddf-b0f8-a49099706a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:02.947425 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d362c0-3817-4ddf-b0f8-a49099706a53 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:02.947715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d362c0-3817-4ddf-b0f8-a49099706a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:02.947972 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d362c0-3817-4ddf-b0f8-a49099706a53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:02.954667 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:02.959019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d362c0-3817-4ddf-b0f8-a49099706a53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:02.961644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d362c0-3817-4ddf-b0f8-a49099706a53 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:02.980188 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-63d362c0-3817-4ddf-b0f8-a49099706a53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:02.980188 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:02.980188 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:02.980188 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:02.980188 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:02.980188 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:02.980188 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:02.980188 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:02.980188 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:02.980188 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:02.980188 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:02.980188 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:02.980188 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:02.980188 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:02.980188 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:02.981140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:02.981140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:02.981140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:02.981140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:02.981140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:02.981140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:02.981140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:02.981140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:02.981140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:02.981140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:02.981140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:02.981140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:02.981140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:02.981140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:02.981140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:02.981140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:02.981140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:02.982024 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:02.982024 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:02.982024 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:02.982024 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:02.982024 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 286/561] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:02.982918 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d362c0-3817-4ddf-b0f8-a49099706a53 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:03.025778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30421a1-3a65-46b1-bb73-deb93e18f614 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:03.062259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c5472-7934-4360-9707-14336aa9d590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:03.063484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c5472-7934-4360-9707-14336aa9d590 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:03.063766 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c5472-7934-4360-9707-14336aa9d590 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:03.064106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c5472-7934-4360-9707-14336aa9d590 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:03.077073 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c5472-7934-4360-9707-14336aa9d590 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:03.078533 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c5472-7934-4360-9707-14336aa9d590 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:03.088278 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d362c0-3817-4ddf-b0f8-a49099706a53 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:03.096692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-8d3c5472-7934-4360-9707-14336aa9d590 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:03.096692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:03.096692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:03.096692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:03.096692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:03.096692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:03.096692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:03.096692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:03.096692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:03.096692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:03.096692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:03.096692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:03.096692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:03.096692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:03.096692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:03.097767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:03.097767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:03.097767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:03.097767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:03.097767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:03.097767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:03.097767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:03.097767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:03.097767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:03.097767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:03.097767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:03.097767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:03.097767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:03.097767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:03.097767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:03.097767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:03.097767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:03.098920 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:03.098920 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:03.098920 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:03.098920 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:03.103199 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 276/562] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:03.104399 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c5472-7934-4360-9707-14336aa9d590 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:03.125684 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd53ef-bc53-46ae-94a4-6148ee82ee9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:03.126478 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd53ef-bc53-46ae-94a4-6148ee82ee9b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:03.126758 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd53ef-bc53-46ae-94a4-6148ee82ee9b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:03.127028 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd53ef-bc53-46ae-94a4-6148ee82ee9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:03.137895 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd53ef-bc53-46ae-94a4-6148ee82ee9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:03.139358 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd53ef-bc53-46ae-94a4-6148ee82ee9b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:03.156853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-c0dd53ef-bc53-46ae-94a4-6148ee82ee9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:03.156853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:03.156853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:03.156853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:03.156853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:03.156853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:03.156853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:03.156853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:03.156853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:03.156853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:03.156853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:03.156853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:03.156853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:03.156853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:03.156853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:03.158016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:03.158016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:03.158016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:03.158016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:03.158016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:03.158016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:03.158016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:03.158016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:03.158016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:03.158016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:03.158016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:03.158016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:03.158016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:03.158016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:03.158016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:03.158016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:03.158016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:03.158920 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:03.158920 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:03.158920 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:03.158920 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:03.158920 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 287/563] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:03.158920 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd53ef-bc53-46ae-94a4-6148ee82ee9b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:03.168757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:03.201717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:03.210799 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c5472-7934-4360-9707-14336aa9d590 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:03.246145 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd61ad29-8e6a-43dc-829d-2f81a16d1fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:03.246937 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61ad29-8e6a-43dc-829d-2f81a16d1fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:03.247249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61ad29-8e6a-43dc-829d-2f81a16d1fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:03.247522 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61ad29-8e6a-43dc-829d-2f81a16d1fe5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:03.258082 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd61ad29-8e6a-43dc-829d-2f81a16d1fe5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:08:03.259729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd53ef-bc53-46ae-94a4-6148ee82ee9b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:03.265834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-dd61ad29-8e6a-43dc-829d-2f81a16d1fe5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:03.265834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:03.265834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:03.265834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:03.265834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:03.265834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:03.265834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:03.265834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:03.265834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:03.265834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:03.265834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:03.265834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:03.265834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:08:03.265834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:08:03.265834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:08:03.265834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:08:03.266944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:08:03.266944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:08:03.266944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:08:03.266944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:08:03.266944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:03.266944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:03.266944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:03.266944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:03.266944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:03.266944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:03.266944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:03.266944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:03.266944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:03.266944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:03.266944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:03.266944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:03.267883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:03.267883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:03.267883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:03.267883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:03.267883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:08:03.267883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:08:03.267883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:08:03.267883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:08:03.267883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:08:03.267883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:08:03.267883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:03.267883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:03.267883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 277/564] 10.222.0.81 () {56 vars in 1102 bytes} [Tue Jun 7 14:08:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:03.268577 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f07a1f-a783-499f-8459-00c67576c0db None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:03.268653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f07a1f-a783-499f-8459-00c67576c0db None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:03.268920 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f07a1f-a783-499f-8459-00c67576c0db None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:03.270474 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 278/565] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:03.271176 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f07a1f-a783-499f-8459-00c67576c0db None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:03.299487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970c9c8-af63-4856-a921-09ac5c2c94e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:03.300147 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970c9c8-af63-4856-a921-09ac5c2c94e6 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:03.300426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970c9c8-af63-4856-a921-09ac5c2c94e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:03.300704 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970c9c8-af63-4856-a921-09ac5c2c94e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:03.301421 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7970c9c8-af63-4856-a921-09ac5c2c94e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:03.301726 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7970c9c8-af63-4856-a921-09ac5c2c94e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:03.303085 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7970c9c8-af63-4856-a921-09ac5c2c94e6 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:03.314788 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 288/566] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:03.315723 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970c9c8-af63-4856-a921-09ac5c2c94e6 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:03.373498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f07a1f-a783-499f-8459-00c67576c0db None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:03.390455 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:03.408752 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a2072-1884-40d0-bbee-894516d3cce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:03.409421 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a2072-1884-40d0-bbee-894516d3cce0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:03.409713 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a2072-1884-40d0-bbee-894516d3cce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:03.409994 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a2072-1884-40d0-bbee-894516d3cce0 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:03.410720 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7a2072-1884-40d0-bbee-894516d3cce0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:03.411028 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7a2072-1884-40d0-bbee-894516d3cce0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:03.412433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7a2072-1884-40d0-bbee-894516d3cce0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:03.416725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970c9c8-af63-4856-a921-09ac5c2c94e6 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:03.418228 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:03.424397 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 279/567] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:03.425207 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a2072-1884-40d0-bbee-894516d3cce0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:03.452394 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014e406-a88f-419a-af4d-0ba8807d9e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:03.453064 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014e406-a88f-419a-af4d-0ba8807d9e32 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:03.453350 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014e406-a88f-419a-af4d-0ba8807d9e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:03.453637 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014e406-a88f-419a-af4d-0ba8807d9e32 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:03.454357 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014e406-a88f-419a-af4d-0ba8807d9e32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:03.454668 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014e406-a88f-419a-af4d-0ba8807d9e32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:03.456084 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014e406-a88f-419a-af4d-0ba8807d9e32 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:03.467430 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 289/568] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:03.468674 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e1658-bef6-424d-acd5-9923a2bf7755 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:03.468967 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e1658-bef6-424d-acd5-9923a2bf7755 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:03.469251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e1658-bef6-424d-acd5-9923a2bf7755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:03.486127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-819e1658-bef6-424d-acd5-9923a2bf7755 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:03.487347 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-819e1658-bef6-424d-acd5-9923a2bf7755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:03.526284 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a2072-1884-40d0-bbee-894516d3cce0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:03.556229 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 290/569] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:08:03.557070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e1658-bef6-424d-acd5-9923a2bf7755 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:03.564456 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4b8a5-e861-4df9-b3f7-b545a1c6705a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:03.565136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4b8a5-e861-4df9-b3f7-b545a1c6705a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:03.565424 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4b8a5-e861-4df9-b3f7-b545a1c6705a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:03.565706 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4b8a5-e861-4df9-b3f7-b545a1c6705a None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:03.566970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4b8a5-e861-4df9-b3f7-b545a1c6705a None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:03.567316 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4b8a5-e861-4df9-b3f7-b545a1c6705a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:03.569042 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4b8a5-e861-4df9-b3f7-b545a1c6705a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:03.578299 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 280/570] 10.222.0.81 () {56 vars in 1103 bytes} [Tue Jun 7 14:08:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:03.579383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4b8a5-e861-4df9-b3f7-b545a1c6705a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:03.623433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:03.655184 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:03.665468 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e1658-bef6-424d-acd5-9923a2bf7755 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:03.700946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ab6bd-9afb-4029-b29e-8651e20ee2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:03.701725 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4b8a5-e861-4df9-b3f7-b545a1c6705a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:03.701843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ab6bd-9afb-4029-b29e-8651e20ee2ba None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:03.702004 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ab6bd-9afb-4029-b29e-8651e20ee2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:03.702274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ab6bd-9afb-4029-b29e-8651e20ee2ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:03.708901 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45ab6bd-9afb-4029-b29e-8651e20ee2ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:03.711143 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45ab6bd-9afb-4029-b29e-8651e20ee2ba None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:03.724826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-f45ab6bd-9afb-4029-b29e-8651e20ee2ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:03.724826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:03.724826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:03.724826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:03.724826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:03.724826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:03.724826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:03.724826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:03.724826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:03.724826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:03.724826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:03.724826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:03.724826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:08:03.724826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:08:03.724826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:08:03.727048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:08:03.727048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:03.727048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:03.727048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:03.727048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:03.727048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:03.727048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:03.727048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:03.727048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:03.727048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:03.727048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:03.727048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:03.727048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:03.727048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:03.727048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:03.727048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:03.727048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:03.728327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:08:03.728327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:08:03.728327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:08:03.728327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:08:03.728327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:08:03.728327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:08:03.728327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:03.728327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:03.728327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 291/571] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:03.728327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ab6bd-9afb-4029-b29e-8651e20ee2ba None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:03.741831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c92b67-f395-4158-8a3a-db44d14b8f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:03.742635 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c92b67-f395-4158-8a3a-db44d14b8f49 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:03.742926 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c92b67-f395-4158-8a3a-db44d14b8f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:03.743212 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c92b67-f395-4158-8a3a-db44d14b8f49 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:03.751374 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c92b67-f395-4158-8a3a-db44d14b8f49 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:03.752809 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c92b67-f395-4158-8a3a-db44d14b8f49 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:03.766896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-c6c92b67-f395-4158-8a3a-db44d14b8f49 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 07 14:08:03.766896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:03.766896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:03.766896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:03.766896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:03.766896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:03.766896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:03.766896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:03.766896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:03.766896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:03.766896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:03.766896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:03.766896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:08:03.766896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:08:03.766896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:08:03.766896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:08:03.768095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:03.768095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:03.768095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:08:03.768095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:03.768095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:08:03.768095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:03.768095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:03.768095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:03.768095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:03.768095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:03.768095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:03.768095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:03.768095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:03.768095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:03.768095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:03.768095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:03.768095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:03.768095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:03.769324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:03.769324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:03.769324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:08:03.769324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:08:03.769324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:08:03.769324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:08:03.769324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:08:03.769324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:08:03.769324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 07 14:08:03.769324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:03.769324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 281/572] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:08:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:03.770268 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c92b67-f395-4158-8a3a-db44d14b8f49 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:03.842671 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ab6bd-9afb-4029-b29e-8651e20ee2ba None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:03.891127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:03.893417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834141f6-302d-4ef0-827f-f591879f7686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:03.894186 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c92b67-f395-4158-8a3a-db44d14b8f49 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:03.894703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834141f6-302d-4ef0-827f-f591879f7686 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:03.895162 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834141f6-302d-4ef0-827f-f591879f7686 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:03.895645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834141f6-302d-4ef0-827f-f591879f7686 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:03.909737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834141f6-302d-4ef0-827f-f591879f7686 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:03.912170 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834141f6-302d-4ef0-827f-f591879f7686 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:03.929518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f01131-e397-4aaa-b1cb-6c315df9192e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:03.930322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f01131-e397-4aaa-b1cb-6c315df9192e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:03.930605 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f01131-e397-4aaa-b1cb-6c315df9192e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:03.930877 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f01131-e397-4aaa-b1cb-6c315df9192e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:03.933512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-834141f6-302d-4ef0-827f-f591879f7686 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:08:03.933512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:03.933512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:03.933512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:03.933512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:03.933512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:03.933512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:03.933512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:03.933512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:03.933512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:03.933512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:03.933512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:03.933512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:08:03.933512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:08:03.933512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:08:03.933512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:08:03.934487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:03.934487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:03.934487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:08:03.934487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:03.934487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:08:03.934487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:03.934487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:03.934487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:03.934487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:03.934487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:03.934487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:03.934487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:03.934487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:03.934487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:03.934487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:03.934487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:03.934487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:03.934487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:03.935482 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:03.935482 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:03.935482 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:08:03.935482 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:08:03.935482 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:08:03.935482 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:08:03.935482 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:08:03.935482 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:08:03.935482 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:08:03.935482 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:03.935482 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 292/573] 10.222.0.81 () {56 vars in 1093 bytes} [Tue Jun 7 14:08:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:03.936675 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834141f6-302d-4ef0-827f-f591879f7686 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:03.962508 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:03.967457 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f01131-e397-4aaa-b1cb-6c315df9192e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:08:03.976250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-55f01131-e397-4aaa-b1cb-6c315df9192e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:03.976250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:03.976250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:03.976250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:03.976250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:03.976250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:03.976250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:03.976250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:03.976250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:03.976250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:03.976250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:03.976250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:03.976250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:08:03.976250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:08:03.976250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:08:03.976250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:08:03.977409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:08:03.977409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:08:03.977409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:08:03.977409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:08:03.977409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:03.977409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:03.977409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:03.977409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:03.977409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:03.977409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:03.977409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:03.977409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:03.977409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:03.977409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:03.977409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:03.977409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:03.983332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:03.983332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:03.983332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:03.983332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:03.983332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:08:03.983332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:08:03.983332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:08:03.983332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:08:03.983332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:08:03.983332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:08:03.983332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:03.983332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:03.983332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 282/574] 10.222.0.81 () {56 vars in 1102 bytes} [Tue Jun 7 14:08:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:03.983332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f01131-e397-4aaa-b1cb-6c315df9192e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:04.078661 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834141f6-302d-4ef0-827f-f591879f7686 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:04.102676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f01131-e397-4aaa-b1cb-6c315df9192e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:04.119208 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab207d29-9186-4c3a-aedb-070b129e817e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:04.120540 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab207d29-9186-4c3a-aedb-070b129e817e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:04.121298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab207d29-9186-4c3a-aedb-070b129e817e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:04.121796 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab207d29-9186-4c3a-aedb-070b129e817e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:04.137670 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9c8e4-0b5f-456d-8d51-159af3a9ebc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:04.138404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9c8e4-0b5f-456d-8d51-159af3a9ebc1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:04.138562 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab207d29-9186-4c3a-aedb-070b129e817e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:04.138680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9c8e4-0b5f-456d-8d51-159af3a9ebc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:04.138957 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9c8e4-0b5f-456d-8d51-159af3a9ebc1 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:04.139782 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f9c8e4-0b5f-456d-8d51-159af3a9ebc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:04.140041 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab207d29-9186-4c3a-aedb-070b129e817e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:04.140132 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f9c8e4-0b5f-456d-8d51-159af3a9ebc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:04.141509 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f9c8e4-0b5f-456d-8d51-159af3a9ebc1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:04.157853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-ab207d29-9186-4c3a-aedb-070b129e817e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:04.157853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:04.157853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:04.157853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:04.157853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:04.157853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:04.157853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:04.157853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:04.157853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:04.157853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:04.157853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:04.157853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:04.157853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:04.157853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:04.157853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:04.158818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:04.158818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:04.158818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:04.158818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:04.158818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:04.158818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:04.158818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:04.158818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:04.158818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:04.158818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:04.158818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:04.158818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:04.158818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:04.158818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:04.158818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:04.158818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:04.158818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:04.159785 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:04.159785 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:04.159785 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:04.159785 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:04.159785 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 293/575] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:04.160075 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab207d29-9186-4c3a-aedb-070b129e817e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:04.169462 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 283/576] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:08:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:04.170313 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9c8e4-0b5f-456d-8d51-159af3a9ebc1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:04.192040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:04.228321 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:04.268948 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab207d29-9186-4c3a-aedb-070b129e817e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:04.273207 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9c8e4-0b5f-456d-8d51-159af3a9ebc1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:04.304976 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be213d-fe4e-49fb-8029-5a8f142f2573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:04.305646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be213d-fe4e-49fb-8029-5a8f142f2573 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:04.305927 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be213d-fe4e-49fb-8029-5a8f142f2573 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:04.306202 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be213d-fe4e-49fb-8029-5a8f142f2573 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:04.307036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be213d-fe4e-49fb-8029-5a8f142f2573 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:04.307384 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be213d-fe4e-49fb-8029-5a8f142f2573 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:04.307531 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07486851-1969-4aeb-b491-5a57f6c99567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:04.308201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07486851-1969-4aeb-b491-5a57f6c99567 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:04.308481 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07486851-1969-4aeb-b491-5a57f6c99567 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:04.308753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07486851-1969-4aeb-b491-5a57f6c99567 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:04.308842 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be213d-fe4e-49fb-8029-5a8f142f2573 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:04.309588 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07486851-1969-4aeb-b491-5a57f6c99567 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:04.309895 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07486851-1969-4aeb-b491-5a57f6c99567 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:04.311329 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07486851-1969-4aeb-b491-5a57f6c99567 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:04.326918 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 294/577] 10.222.0.81 () {56 vars in 1137 bytes} [Tue Jun 7 14:08:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:04.327797 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be213d-fe4e-49fb-8029-5a8f142f2573 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:04.329499 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 284/578] 10.222.0.81 () {56 vars in 1133 bytes} [Tue Jun 7 14:08:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:04.330319 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07486851-1969-4aeb-b491-5a57f6c99567 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:04.412478 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:04.436510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be213d-fe4e-49fb-8029-5a8f142f2573 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:04.437141 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07486851-1969-4aeb-b491-5a57f6c99567 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:04.462367 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:04.474441 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84d95f-5e71-4628-8d23-ff26b1736fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:04.475148 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84d95f-5e71-4628-8d23-ff26b1736fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:04.475516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84d95f-5e71-4628-8d23-ff26b1736fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:04.476025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84d95f-5e71-4628-8d23-ff26b1736fb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:04.477424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd84d95f-5e71-4628-8d23-ff26b1736fb3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:04.477424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd84d95f-5e71-4628-8d23-ff26b1736fb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:04.478000 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3bfe6-6ef0-4488-a65b-e3b7afa24d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:04.478660 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd84d95f-5e71-4628-8d23-ff26b1736fb3 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:04.478768 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3bfe6-6ef0-4488-a65b-e3b7afa24d00 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:04.478957 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3bfe6-6ef0-4488-a65b-e3b7afa24d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:04.479236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3bfe6-6ef0-4488-a65b-e3b7afa24d00 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:04.479975 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3bfe6-6ef0-4488-a65b-e3b7afa24d00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:04.480282 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3bfe6-6ef0-4488-a65b-e3b7afa24d00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:04.481640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3bfe6-6ef0-4488-a65b-e3b7afa24d00 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:04.486875 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 295/579] 10.222.0.81 () {56 vars in 1103 bytes} [Tue Jun 7 14:08:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:04.487760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84d95f-5e71-4628-8d23-ff26b1736fb3 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:04.495273 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 285/580] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:04.496346 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3bfe6-6ef0-4488-a65b-e3b7afa24d00 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:04.596185 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84d95f-5e71-4628-8d23-ff26b1736fb3 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:04.601899 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3bfe6-6ef0-4488-a65b-e3b7afa24d00 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:04.635271 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22b5e6-9747-4569-95c8-df3373f61016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:04.636019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22b5e6-9747-4569-95c8-df3373f61016 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:04.636465 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22b5e6-9747-4569-95c8-df3373f61016 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:04.636595 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ccf251-aeb0-457f-9dbd-f90e0ad27fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:04.636804 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22b5e6-9747-4569-95c8-df3373f61016 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:04.637270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccf251-aeb0-457f-9dbd-f90e0ad27fcc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:04.637555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccf251-aeb0-457f-9dbd-f90e0ad27fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:04.637642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22b5e6-9747-4569-95c8-df3373f61016 None admin] RBAC: Adding query filter params (role.id=a72e5464b7b746978805326c224753bc, scope.project.id=f67bad575f8b4a47bee7c314497cb2fe, user.id=da7c0f99060a44eaab743ba71f159e88) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:04.637834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccf251-aeb0-457f-9dbd-f90e0ad27fcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:04.637932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22b5e6-9747-4569-95c8-df3373f61016 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:04.639941 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22b5e6-9747-4569-95c8-df3373f61016 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:04.645428 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:04.645934 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 296/581] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:08:04 2022] GET /identity/v3/role_assignments?user.id=da7c0f99060a44eaab743ba71f159e88&scope.project.id=f67bad575f8b4a47bee7c314497cb2fe&role.id=a72e5464b7b746978805326c224753bc => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:04.646759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22b5e6-9747-4569-95c8-df3373f61016 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:04.652872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ccf251-aeb0-457f-9dbd-f90e0ad27fcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:04.654372 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ccf251-aeb0-457f-9dbd-f90e0ad27fcc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:04.673745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-c6ccf251-aeb0-457f-9dbd-f90e0ad27fcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:04.673745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:04.673745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:04.673745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:04.673745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:04.673745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:04.673745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:04.673745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:04.673745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:04.673745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:04.673745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:04.673745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:04.673745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:04.673745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:04.673745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:04.675023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:04.675023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:04.675023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:04.675023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:04.675023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:04.675023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:04.675023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:04.675023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:04.675023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:04.675023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:04.675023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:04.675023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:04.675023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:04.675023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:04.675023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:04.675023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:04.675023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:04.683007 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:04.683007 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:04.683007 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:04.683007 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:04.683007 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 286/582] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:04.683007 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ccf251-aeb0-457f-9dbd-f90e0ad27fcc None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:04.727587 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:04.795455 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ccf251-aeb0-457f-9dbd-f90e0ad27fcc None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:04.796013 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22b5e6-9747-4569-95c8-df3373f61016 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:04.834448 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472cab0-7ae5-4fc8-911a-e90088090b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:04.836346 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472cab0-7ae5-4fc8-911a-e90088090b8f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:04.836658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472cab0-7ae5-4fc8-911a-e90088090b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:04.836932 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472cab0-7ae5-4fc8-911a-e90088090b8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:04.840727 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f1c6c-0b13-4f77-9018-a90373ca8d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:04.841538 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f1c6c-0b13-4f77-9018-a90373ca8d24 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:04.841824 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f1c6c-0b13-4f77-9018-a90373ca8d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:04.842103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f1c6c-0b13-4f77-9018-a90373ca8d24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:04.848872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2472cab0-7ae5-4fc8-911a-e90088090b8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:04.850314 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2472cab0-7ae5-4fc8-911a-e90088090b8f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:04.858205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f1c6c-0b13-4f77-9018-a90373ca8d24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:04.859707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f1c6c-0b13-4f77-9018-a90373ca8d24 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:04.879872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-2472cab0-7ae5-4fc8-911a-e90088090b8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:04.879872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:04.879872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:04.879872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:04.879872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:04.879872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:04.879872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:04.879872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:04.879872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:04.879872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:04.879872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:04.879872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:04.879872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:04.879872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:04.879872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:04.881019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:04.881019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:04.881019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:04.881019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:04.881019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:04.881019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:04.881019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:04.881019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:04.881019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:04.881019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:04.881019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:04.881019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:04.881019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:04.881019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:04.881019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:04.881019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:04.881019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:04.882631 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:04.882631 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:04.882631 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:04.882631 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:04.882870 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 287/583] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:04.887695 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-647f1c6c-0b13-4f77-9018-a90373ca8d24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:04.887695 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:04.887695 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:04.887695 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:04.887695 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:04.887695 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:04.887695 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:04.887695 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:04.887695 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:04.887695 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:04.887695 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:04.887695 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:04.887695 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:04.887695 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:04.887695 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:04.888801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:04.888801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:04.888801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:04.888801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:04.888801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:04.888801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:04.888801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:04.888801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:04.888801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:04.888801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:04.888801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:04.888801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:04.888801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:04.888801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:04.888801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:04.888801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:04.888801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:04.889728 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:04.889728 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:04.889728 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:04.889728 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:04.889957 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472cab0-7ae5-4fc8-911a-e90088090b8f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:04.890031 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 297/584] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:04.890031 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f1c6c-0b13-4f77-9018-a90373ca8d24 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:04.946940 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:05.001356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:05.004406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f1c6c-0b13-4f77-9018-a90373ca8d24 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:05.022889 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472cab0-7ae5-4fc8-911a-e90088090b8f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:05.041113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8793a25e-ceec-46a7-982b-d1b415e277fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:05.041877 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793a25e-ceec-46a7-982b-d1b415e277fc None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:05.042229 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793a25e-ceec-46a7-982b-d1b415e277fc None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:05.042571 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793a25e-ceec-46a7-982b-d1b415e277fc None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:05.043423 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8793a25e-ceec-46a7-982b-d1b415e277fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:05.043793 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8793a25e-ceec-46a7-982b-d1b415e277fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:05.045219 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8793a25e-ceec-46a7-982b-d1b415e277fc None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:05.062636 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 298/585] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:05.062636 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8793a25e-ceec-46a7-982b-d1b415e277fc None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:05.077362 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fec46e-3693-4678-9d9b-5070fc300688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:05.078738 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fec46e-3693-4678-9d9b-5070fc300688 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:05.079305 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fec46e-3693-4678-9d9b-5070fc300688 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:05.079985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fec46e-3693-4678-9d9b-5070fc300688 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:05.092409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fec46e-3693-4678-9d9b-5070fc300688 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:05.095020 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fec46e-3693-4678-9d9b-5070fc300688 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:05.115681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-a7fec46e-3693-4678-9d9b-5070fc300688 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 14:08:05.115681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:05.115681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:05.115681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:05.115681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:05.115681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:05.115681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:05.115681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:05.115681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:05.115681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:05.115681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:05.115681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:05.115681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:08:05.115681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:08:05.115681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:08:05.115681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:08:05.117847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:05.117847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:05.117847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:08:05.117847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:05.117847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:08:05.117847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:05.117847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:05.117847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:05.117847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:05.117847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:05.117847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:05.117847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:05.117847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:05.117847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:05.117847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:05.117847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:05.117847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:05.117847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:05.119003 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:05.119003 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:05.119003 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:08:05.119003 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:08:05.119003 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:08:05.119003 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:08:05.119003 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:08:05.119003 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:08:05.119003 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 14:08:05.119003 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:05.124655 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 288/586] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:08:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:05.124655 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fec46e-3693-4678-9d9b-5070fc300688 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:05.177580 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8793a25e-ceec-46a7-982b-d1b415e277fc None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:05.216974 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d9dd2-e881-4e4f-ad2d-241d3fa08f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:05.217878 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d9dd2-e881-4e4f-ad2d-241d3fa08f3a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:05.218307 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d9dd2-e881-4e4f-ad2d-241d3fa08f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:05.218853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d9dd2-e881-4e4f-ad2d-241d3fa08f3a None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:05.219888 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d9dd2-e881-4e4f-ad2d-241d3fa08f3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:05.220355 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d9dd2-e881-4e4f-ad2d-241d3fa08f3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:05.221934 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d9dd2-e881-4e4f-ad2d-241d3fa08f3a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:05.235272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fec46e-3693-4678-9d9b-5070fc300688 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:05.239091 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:05.239207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 299/587] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:05.240015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d9dd2-e881-4e4f-ad2d-241d3fa08f3a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:05.265200 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:05.272938 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f5335-047b-4cb3-b898-c331d5cd25f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:05.273625 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f5335-047b-4cb3-b898-c331d5cd25f3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:05.274064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f5335-047b-4cb3-b898-c331d5cd25f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:05.274344 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f5335-047b-4cb3-b898-c331d5cd25f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:05.275073 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820f5335-047b-4cb3-b898-c331d5cd25f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:05.275420 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820f5335-047b-4cb3-b898-c331d5cd25f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:05.276787 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820f5335-047b-4cb3-b898-c331d5cd25f3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:05.293792 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 289/588] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:05.293792 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f5335-047b-4cb3-b898-c331d5cd25f3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:05.364636 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d9dd2-e881-4e4f-ad2d-241d3fa08f3a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:05.404573 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749353d4-d56f-4d47-b8d3-94eec8f7e17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:05.405376 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749353d4-d56f-4d47-b8d3-94eec8f7e17c None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:05.405665 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749353d4-d56f-4d47-b8d3-94eec8f7e17c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:05.405943 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749353d4-d56f-4d47-b8d3-94eec8f7e17c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:05.409081 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f5335-047b-4cb3-b898-c331d5cd25f3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:05.414523 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749353d4-d56f-4d47-b8d3-94eec8f7e17c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:05.416114 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749353d4-d56f-4d47-b8d3-94eec8f7e17c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:05.432120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-749353d4-d56f-4d47-b8d3-94eec8f7e17c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:05.432120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:05.432120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:05.432120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:05.432120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:05.432120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:05.432120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:05.432120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:05.432120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:05.432120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:05.432120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:05.432120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:05.432120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:08:05.432120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:08:05.432120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:08:05.433061 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:08:05.433061 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:05.433061 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:05.433061 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:05.433061 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:05.433061 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:05.433061 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:05.433061 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:05.433061 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:05.433061 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:05.433061 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:05.433061 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:05.433061 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:05.433061 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:05.433061 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:05.433061 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:05.433061 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:05.434067 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:08:05.434067 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:08:05.434067 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:08:05.434067 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:08:05.434067 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:08:05.434067 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:08:05.434067 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:05.434067 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:05.434067 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 300/589] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:05.434570 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749353d4-d56f-4d47-b8d3-94eec8f7e17c None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:05.454025 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83671e5f-e08b-4d71-bd83-3f8231841046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:05.454697 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83671e5f-e08b-4d71-bd83-3f8231841046 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:05.454974 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83671e5f-e08b-4d71-bd83-3f8231841046 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:05.455329 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83671e5f-e08b-4d71-bd83-3f8231841046 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:05.456194 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83671e5f-e08b-4d71-bd83-3f8231841046 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:05.456497 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83671e5f-e08b-4d71-bd83-3f8231841046 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:05.457912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83671e5f-e08b-4d71-bd83-3f8231841046 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:05.481640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 290/590] 10.222.0.81 () {56 vars in 1137 bytes} [Tue Jun 7 14:08:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:05.481640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83671e5f-e08b-4d71-bd83-3f8231841046 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:05.508618 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:05.527950 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:05.547682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749353d4-d56f-4d47-b8d3-94eec8f7e17c None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:05.590228 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927e84f-3a99-4119-8c34-adb79e6ba716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:05.591026 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927e84f-3a99-4119-8c34-adb79e6ba716 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:05.591327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927e84f-3a99-4119-8c34-adb79e6ba716 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:05.591602 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927e84f-3a99-4119-8c34-adb79e6ba716 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:05.599500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927e84f-3a99-4119-8c34-adb79e6ba716 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:05.601825 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927e84f-3a99-4119-8c34-adb79e6ba716 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:05.606326 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83671e5f-e08b-4d71-bd83-3f8231841046 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:05.619087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-4927e84f-3a99-4119-8c34-adb79e6ba716 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 14:08:05.619087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:05.619087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:05.619087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:05.619087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:05.619087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:05.619087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:05.619087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:05.619087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:05.619087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:05.619087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:05.619087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:05.619087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:08:05.619087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:08:05.619087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:08:05.619087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:08:05.620324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:05.620324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:05.620324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:08:05.620324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:05.620324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:08:05.620324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:05.620324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:05.620324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:05.620324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:05.620324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:05.620324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:05.620324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:05.620324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:05.620324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:05.620324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:05.620324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:05.620324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:05.620324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:05.621700 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:05.621700 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:05.621700 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:08:05.621700 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:08:05.621700 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:08:05.621700 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:08:05.621700 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:08:05.621700 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:08:05.621700 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 14:08:05.621700 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:05.621700 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 301/591] 10.222.0.81 () {56 vars in 1102 bytes} [Tue Jun 7 14:08:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:05.621700 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927e84f-3a99-4119-8c34-adb79e6ba716 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:05.654559 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ea387-2420-4e82-bc89-d37aa586b5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:05.655696 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ea387-2420-4e82-bc89-d37aa586b5ae None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:05.656142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ea387-2420-4e82-bc89-d37aa586b5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:05.656421 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ea387-2420-4e82-bc89-d37aa586b5ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:05.664894 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61ea387-2420-4e82-bc89-d37aa586b5ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:05.666402 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61ea387-2420-4e82-bc89-d37aa586b5ae None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:05.681162 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-e61ea387-2420-4e82-bc89-d37aa586b5ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:05.681162 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:05.681162 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:05.681162 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:05.681162 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:05.681162 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:05.681162 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:05.681162 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:05.681162 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:05.681162 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:05.681162 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:05.681162 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:05.681162 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:08:05.681162 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:08:05.681162 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:08:05.682055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:08:05.682055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:05.682055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:05.682055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:05.682055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:05.682055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:05.682055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:05.682055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:05.682055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:05.682055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:05.682055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:05.682055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:05.682055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:05.682055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:05.682055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:05.682055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:05.682055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:05.682954 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:08:05.682954 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:08:05.682954 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:08:05.682954 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:08:05.682954 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:08:05.682954 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:08:05.682954 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:05.682954 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:05.682954 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 291/592] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:05.684078 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ea387-2420-4e82-bc89-d37aa586b5ae None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:05.736162 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927e84f-3a99-4119-8c34-adb79e6ba716 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:05.764773 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:05.774688 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a99f06-9c80-445d-95be-fb4e3c930ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:05.775650 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a99f06-9c80-445d-95be-fb4e3c930ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:05.776082 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a99f06-9c80-445d-95be-fb4e3c930ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:05.776507 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a99f06-9c80-445d-95be-fb4e3c930ad7 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:05.777436 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a99f06-9c80-445d-95be-fb4e3c930ad7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:05.777911 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a99f06-9c80-445d-95be-fb4e3c930ad7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:05.779493 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a99f06-9c80-445d-95be-fb4e3c930ad7 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:05.785493 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ea387-2420-4e82-bc89-d37aa586b5ae None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:05.790039 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:05.798657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 302/593] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:08:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:05.798657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a99f06-9c80-445d-95be-fb4e3c930ad7 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:05.826342 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a6b200-db1f-48ff-a728-daa8f58a8ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:05.827147 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6b200-db1f-48ff-a728-daa8f58a8ebf None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:05.827462 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6b200-db1f-48ff-a728-daa8f58a8ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:05.827739 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6b200-db1f-48ff-a728-daa8f58a8ebf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:05.841138 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a6b200-db1f-48ff-a728-daa8f58a8ebf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:05.842592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a6b200-db1f-48ff-a728-daa8f58a8ebf None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:05.860910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-91a6b200-db1f-48ff-a728-daa8f58a8ebf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:05.860910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:05.860910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:05.860910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:05.860910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:05.860910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:05.860910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:05.860910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:05.860910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:05.860910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:05.860910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:05.860910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:05.860910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:05.860910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:05.860910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:05.861842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:05.861842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:05.861842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:05.861842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:05.861842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:05.861842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:05.861842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:05.861842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:05.861842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:05.861842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:05.861842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:05.861842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:05.861842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:05.861842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:05.861842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:05.861842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:05.861842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:05.862749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:05.862749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:05.862749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:05.862749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:05.862749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 292/594] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:05.863584 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a6b200-db1f-48ff-a728-daa8f58a8ebf None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:05.914700 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a99f06-9c80-445d-95be-fb4e3c930ad7 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:05.951395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75be77-54ed-4ce9-960a-962dea4224f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:05.952351 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75be77-54ed-4ce9-960a-962dea4224f4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:05.952638 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75be77-54ed-4ce9-960a-962dea4224f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:05.952923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75be77-54ed-4ce9-960a-962dea4224f4 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:05.954739 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75be77-54ed-4ce9-960a-962dea4224f4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:05.955063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75be77-54ed-4ce9-960a-962dea4224f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:05.956532 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75be77-54ed-4ce9-960a-962dea4224f4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:05.977401 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 303/595] 10.222.0.81 () {56 vars in 1139 bytes} [Tue Jun 7 14:08:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:05.978197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75be77-54ed-4ce9-960a-962dea4224f4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:05.992470 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a6b200-db1f-48ff-a728-daa8f58a8ebf None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:06.025633 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:06.032474 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa760e3-8741-4b83-aa37-81cde664e19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:06.033422 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa760e3-8741-4b83-aa37-81cde664e19d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:06.033711 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa760e3-8741-4b83-aa37-81cde664e19d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:06.034155 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa760e3-8741-4b83-aa37-81cde664e19d None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:06.035309 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa760e3-8741-4b83-aa37-81cde664e19d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:06.035643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa760e3-8741-4b83-aa37-81cde664e19d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:06.037053 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa760e3-8741-4b83-aa37-81cde664e19d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:06.044442 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:06.055641 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 293/596] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:08:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:06.055641 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa760e3-8741-4b83-aa37-81cde664e19d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:06.093224 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75be77-54ed-4ce9-960a-962dea4224f4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:06.130561 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60db57-8bcf-4bc8-b400-b8cc066e96d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:06.132159 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60db57-8bcf-4bc8-b400-b8cc066e96d1 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:06.132439 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60db57-8bcf-4bc8-b400-b8cc066e96d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:06.132714 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60db57-8bcf-4bc8-b400-b8cc066e96d1 None admin] PATH_INFO: `/v3/projects/f67bad575f8b4a47bee7c314497cb2fe/users/b6180393da0f495db964f0b2588d8b90/roles/a72e5464b7b746978805326c224753bc` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:06.147091 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60db57-8bcf-4bc8-b400-b8cc066e96d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=f67bad575f8b4a47bee7c314497cb2fe, user_id=b6180393da0f495db964f0b2588d8b90, role_id=a72e5464b7b746978805326c224753bc)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:06.148751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60db57-8bcf-4bc8-b400-b8cc066e96d1 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:06.158037 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa760e3-8741-4b83-aa37-81cde664e19d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:06.172378 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 304/597] 10.222.0.81 () {58 vars in 1439 bytes} [Tue Jun 7 14:08:05 2022] PUT /identity/v3/projects/f67bad575f8b4a47bee7c314497cb2fe/users/b6180393da0f495db964f0b2588d8b90/roles/a72e5464b7b746978805326c224753bc => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:08:06.173583 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60db57-8bcf-4bc8-b400-b8cc066e96d1 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:06.196599 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bda5f1-5e81-48eb-9b6f-e3a1b9734993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:06.197284 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bda5f1-5e81-48eb-9b6f-e3a1b9734993 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:06.197568 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bda5f1-5e81-48eb-9b6f-e3a1b9734993 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:06.197896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bda5f1-5e81-48eb-9b6f-e3a1b9734993 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:06.198743 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bda5f1-5e81-48eb-9b6f-e3a1b9734993 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:06.199058 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bda5f1-5e81-48eb-9b6f-e3a1b9734993 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:06.200517 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bda5f1-5e81-48eb-9b6f-e3a1b9734993 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:06.214541 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 294/598] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:06.215484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bda5f1-5e81-48eb-9b6f-e3a1b9734993 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:06.277979 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:06.329823 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60db57-8bcf-4bc8-b400-b8cc066e96d1 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:06.340799 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:06.342702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bda5f1-5e81-48eb-9b6f-e3a1b9734993 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:06.376656 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dae24d-4ab8-48f3-ad33-95041c7a6b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:06.377948 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dae24d-4ab8-48f3-ad33-95041c7a6b8b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:06.378436 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dae24d-4ab8-48f3-ad33-95041c7a6b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:06.378931 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dae24d-4ab8-48f3-ad33-95041c7a6b8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:06.390266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33388a02-25e0-4fb4-966f-acfa81348a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:06.391479 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33388a02-25e0-4fb4-966f-acfa81348a41 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:06.391767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33388a02-25e0-4fb4-966f-acfa81348a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:06.392043 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33388a02-25e0-4fb4-966f-acfa81348a41 None admin] PATH_INFO: `/v3/projects/f67bad575f8b4a47bee7c314497cb2fe/users/2ae726fbc9cf40e68c7587552a118a7a/roles/a72e5464b7b746978805326c224753bc` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:06.392882 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dae24d-4ab8-48f3-ad33-95041c7a6b8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:06.394724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dae24d-4ab8-48f3-ad33-95041c7a6b8b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:06.411347 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33388a02-25e0-4fb4-966f-acfa81348a41 None admin] RBAC: Authorizing `identity:create_grant(project_id=f67bad575f8b4a47bee7c314497cb2fe, user_id=2ae726fbc9cf40e68c7587552a118a7a, role_id=a72e5464b7b746978805326c224753bc)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:06.412786 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33388a02-25e0-4fb4-966f-acfa81348a41 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:06.421812 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-12dae24d-4ab8-48f3-ad33-95041c7a6b8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:06.421812 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:06.421812 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:06.421812 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:06.421812 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:06.421812 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:06.421812 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:06.421812 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:06.421812 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:06.421812 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:06.421812 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:06.421812 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:06.421812 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:06.421812 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:06.421812 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:06.422715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:06.422715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:06.422715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:06.422715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:06.422715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:06.422715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:06.422715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:06.422715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:06.422715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:06.422715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:06.422715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:06.422715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:06.422715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:06.422715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:06.422715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:06.422715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:06.422715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:06.423710 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:06.423710 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:06.423710 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:06.423710 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:06.425531 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 305/599] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:06.426618 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dae24d-4ab8-48f3-ad33-95041c7a6b8b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:06.431124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 295/600] 10.222.0.81 () {58 vars in 1439 bytes} [Tue Jun 7 14:08:06 2022] PUT /identity/v3/projects/f67bad575f8b4a47bee7c314497cb2fe/users/2ae726fbc9cf40e68c7587552a118a7a/roles/a72e5464b7b746978805326c224753bc => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:08:06.432507 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33388a02-25e0-4fb4-966f-acfa81348a41 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:06.547911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33388a02-25e0-4fb4-966f-acfa81348a41 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:06.549578 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:06.566531 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dae24d-4ab8-48f3-ad33-95041c7a6b8b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:06.586642 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a81234-30fc-4414-9917-f07d5099693f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:06.587436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a81234-30fc-4414-9917-f07d5099693f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:06.587728 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a81234-30fc-4414-9917-f07d5099693f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:06.588004 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a81234-30fc-4414-9917-f07d5099693f None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:06.588743 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a81234-30fc-4414-9917-f07d5099693f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:06.589048 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a81234-30fc-4414-9917-f07d5099693f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:06.590556 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a81234-30fc-4414-9917-f07d5099693f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:06.604807 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 296/601] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:06.611900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a81234-30fc-4414-9917-f07d5099693f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:06.614169 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625ad06-fb6e-440e-b553-c753d998df64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:06.615000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625ad06-fb6e-440e-b553-c753d998df64 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:06.616546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625ad06-fb6e-440e-b553-c753d998df64 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:06.616546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625ad06-fb6e-440e-b553-c753d998df64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:06.633286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625ad06-fb6e-440e-b553-c753d998df64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:06.634814 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625ad06-fb6e-440e-b553-c753d998df64 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:06.655048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-9625ad06-fb6e-440e-b553-c753d998df64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:06.655048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:06.655048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:06.655048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:06.655048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:06.655048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:06.655048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:06.655048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:06.655048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:06.655048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:06.655048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:06.655048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:06.655048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:08:06.655048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:08:06.655048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:08:06.657356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:08:06.657356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:06.657356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:06.657356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:06.657356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:06.657356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:06.657356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:06.657356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:06.657356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:06.657356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:06.657356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:06.657356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:06.657356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:06.657356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:06.657356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:06.657356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:06.657356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:06.660991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:08:06.660991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:08:06.660991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:08:06.660991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:08:06.660991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:08:06.660991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:08:06.660991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:06.660991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:06.660991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 306/602] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:06.662999 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625ad06-fb6e-440e-b553-c753d998df64 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:06.710751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:06.847540 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a81234-30fc-4414-9917-f07d5099693f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:06.879052 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625ad06-fb6e-440e-b553-c753d998df64 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:06.940311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7fb3e-87e0-4e44-a13e-6012afecf7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:06.941131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7fb3e-87e0-4e44-a13e-6012afecf7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:06.941522 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7fb3e-87e0-4e44-a13e-6012afecf7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:06.941808 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7fb3e-87e0-4e44-a13e-6012afecf7c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:06.953357 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5baea-18f4-4cec-80ee-98c7e105b0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:06.954028 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5baea-18f4-4cec-80ee-98c7e105b0ed None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:06.954309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5baea-18f4-4cec-80ee-98c7e105b0ed None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:06.954580 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5baea-18f4-4cec-80ee-98c7e105b0ed None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:06.955402 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5baea-18f4-4cec-80ee-98c7e105b0ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:06.955712 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5baea-18f4-4cec-80ee-98c7e105b0ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:06.957143 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5baea-18f4-4cec-80ee-98c7e105b0ed None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:06.961327 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7fb3e-87e0-4e44-a13e-6012afecf7c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:06.963944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7fb3e-87e0-4e44-a13e-6012afecf7c6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:06.990326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 307/603] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:08:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:06.990676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-64b7fb3e-87e0-4e44-a13e-6012afecf7c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:06.990676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:06.990676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:06.990676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:06.990676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:06.990676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:06.990676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:06.990676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:06.990676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:06.990676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:06.990676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:06.990676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:06.990676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:08:06.990676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:08:06.990676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:08:06.991971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:08:06.991971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:06.991971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:06.991971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:06.991971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:06.991971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:06.991971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:06.991971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:06.991971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:06.991971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:06.991971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:06.991971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:06.991971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:06.991971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:06.991971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:06.991971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:06.991971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:06.992887 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:08:06.992887 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:08:06.992887 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:08:06.992887 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:08:06.992887 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:08:06.992887 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:08:06.992887 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:06.992887 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:06.992887 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 297/604] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:06.993378 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5baea-18f4-4cec-80ee-98c7e105b0ed None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:06.998310 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7fb3e-87e0-4e44-a13e-6012afecf7c6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:07.049348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:07.123401 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:07.157771 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5baea-18f4-4cec-80ee-98c7e105b0ed None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:07.199377 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7fb3e-87e0-4e44-a13e-6012afecf7c6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:07.203560 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b248d0a-2db2-4f4a-a69f-0f016f341f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:07.204464 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b248d0a-2db2-4f4a-a69f-0f016f341f39 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:07.204748 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b248d0a-2db2-4f4a-a69f-0f016f341f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:07.205024 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b248d0a-2db2-4f4a-a69f-0f016f341f39 None admin] PATH_INFO: `/v3/projects/f67bad575f8b4a47bee7c314497cb2fe/users/38b1ff21b8e5445b878740a327e4442f/roles/a72e5464b7b746978805326c224753bc` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:07.227747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b248d0a-2db2-4f4a-a69f-0f016f341f39 None admin] RBAC: Authorizing `identity:create_grant(project_id=f67bad575f8b4a47bee7c314497cb2fe, user_id=38b1ff21b8e5445b878740a327e4442f, role_id=a72e5464b7b746978805326c224753bc)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:07.229547 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b248d0a-2db2-4f4a-a69f-0f016f341f39 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:07.246017 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1a09e-ca94-4ac5-b1eb-bb82c6267613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:07.246691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1a09e-ca94-4ac5-b1eb-bb82c6267613 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:07.246969 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1a09e-ca94-4ac5-b1eb-bb82c6267613 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:07.248001 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1a09e-ca94-4ac5-b1eb-bb82c6267613 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:07.248109 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1a09e-ca94-4ac5-b1eb-bb82c6267613 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:07.248353 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1a09e-ca94-4ac5-b1eb-bb82c6267613 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:07.249927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1a09e-ca94-4ac5-b1eb-bb82c6267613 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:07.253289 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 308/605] 10.222.0.81 () {58 vars in 1439 bytes} [Tue Jun 7 14:08:06 2022] PUT /identity/v3/projects/f67bad575f8b4a47bee7c314497cb2fe/users/38b1ff21b8e5445b878740a327e4442f/roles/a72e5464b7b746978805326c224753bc => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:08:07.255083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0fde32-b02f-4d73-b0bd-87b8b30086cc None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:07.255692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0fde32-b02f-4d73-b0bd-87b8b30086cc None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:07.256190 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0fde32-b02f-4d73-b0bd-87b8b30086cc None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:07.258927 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 309/606] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:07.270100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae62d80-57c4-401a-a156-e38f06a0a09d None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:07.270743 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae62d80-57c4-401a-a156-e38f06a0a09d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:07.271246 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae62d80-57c4-401a-a156-e38f06a0a09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:07.287800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 298/607] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:08:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:07.294333 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-bae62d80-57c4-401a-a156-e38f06a0a09d None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:07.295920 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-bae62d80-57c4-401a-a156-e38f06a0a09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:07.297110 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1a09e-ca94-4ac5-b1eb-bb82c6267613 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:07.440681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:07.467315 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1a09e-ca94-4ac5-b1eb-bb82c6267613 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:07.497124 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 310/608] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:08:07.510051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65331a66-712f-452a-8425-723f39dccc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:07.511349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65331a66-712f-452a-8425-723f39dccc8d None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:07.511841 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65331a66-712f-452a-8425-723f39dccc8d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:07.512350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65331a66-712f-452a-8425-723f39dccc8d None admin] PATH_INFO: `/v3/projects/f67bad575f8b4a47bee7c314497cb2fe/users/a6a08eaecbcd41228571739d8387882a/roles/a72e5464b7b746978805326c224753bc` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:07.529596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65331a66-712f-452a-8425-723f39dccc8d None admin] RBAC: Authorizing `identity:create_grant(project_id=f67bad575f8b4a47bee7c314497cb2fe, user_id=a6a08eaecbcd41228571739d8387882a, role_id=a72e5464b7b746978805326c224753bc)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:07.531417 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65331a66-712f-452a-8425-723f39dccc8d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:07.535364 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae62d80-57c4-401a-a156-e38f06a0a09d None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:07.548680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 299/609] 10.222.0.81 () {58 vars in 1439 bytes} [Tue Jun 7 14:08:07 2022] PUT /identity/v3/projects/f67bad575f8b4a47bee7c314497cb2fe/users/a6a08eaecbcd41228571739d8387882a/roles/a72e5464b7b746978805326c224753bc => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:08:07.623660 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:07.682930 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae62d80-57c4-401a-a156-e38f06a0a09d None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:07.739840 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c3474-6bad-4e21-9fe3-696cbf29b844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:07.741649 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c3474-6bad-4e21-9fe3-696cbf29b844 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:07.742387 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c3474-6bad-4e21-9fe3-696cbf29b844 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:07.743117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c3474-6bad-4e21-9fe3-696cbf29b844 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:07.758916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c3474-6bad-4e21-9fe3-696cbf29b844 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:08:07.768959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-809c3474-6bad-4e21-9fe3-696cbf29b844 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:07.768959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:07.768959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:07.768959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:07.768959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:07.768959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:07.768959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:07.768959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:07.768959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:07.768959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:07.768959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:07.768959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:07.768959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:08:07.768959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:08:07.768959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:08:07.768959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:08:07.770574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:08:07.770574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:08:07.770574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:08:07.770574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:08:07.770574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:07.770574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:07.770574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:07.770574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:07.770574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:07.770574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:07.770574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:07.770574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:07.770574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:07.770574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:07.770574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:07.770574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:07.771472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:07.771472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:07.771472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:07.771472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:07.771472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:08:07.771472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:08:07.771472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:08:07.771472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:08:07.771472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:08:07.771472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:08:07.771472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:07.771472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:07.775249 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 311/610] 10.222.0.81 () {56 vars in 1102 bytes} [Tue Jun 7 14:08:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:07.784454 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65331a66-712f-452a-8425-723f39dccc8d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:07.917557 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65331a66-712f-452a-8425-723f39dccc8d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:07.956307 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:07.966273 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcf605-4a33-4711-9790-2ca8367cd819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:07.967382 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcf605-4a33-4711-9790-2ca8367cd819 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:07.968065 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcf605-4a33-4711-9790-2ca8367cd819 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:07.968341 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcf605-4a33-4711-9790-2ca8367cd819 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:07.969113 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fcf605-4a33-4711-9790-2ca8367cd819 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:07.969416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fcf605-4a33-4711-9790-2ca8367cd819 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:07.970954 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fcf605-4a33-4711-9790-2ca8367cd819 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:07.980726 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 300/611] 10.222.0.81 () {56 vars in 1103 bytes} [Tue Jun 7 14:08:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:07.986591 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c3474-6bad-4e21-9fe3-696cbf29b844 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:08.115492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c3474-6bad-4e21-9fe3-696cbf29b844 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:08.142830 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:08.170850 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e473e-66d6-4e0a-b69c-188560d82764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:08.172141 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e473e-66d6-4e0a-b69c-188560d82764 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:08.172426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e473e-66d6-4e0a-b69c-188560d82764 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:08.172701 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e473e-66d6-4e0a-b69c-188560d82764 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:08.192759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e473e-66d6-4e0a-b69c-188560d82764 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:08.194829 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e473e-66d6-4e0a-b69c-188560d82764 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:08.219163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-5e6e473e-66d6-4e0a-b69c-188560d82764 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:08.219163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:08.219163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:08.219163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:08.219163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:08.219163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:08.219163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:08.219163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:08.219163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:08.219163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:08.219163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:08.219163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:08.219163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:08.219163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:08.219163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:08.220153 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:08.220153 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:08.220153 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:08.220153 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:08.220153 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:08.220153 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:08.220153 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:08.220153 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:08.220153 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:08.220153 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:08.220153 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:08.220153 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:08.220153 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:08.220153 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:08.220153 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:08.220153 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:08.220153 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:08.221063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:08.221063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:08.221063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:08.221063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:08.221786 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 312/612] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:08.230298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcf605-4a33-4711-9790-2ca8367cd819 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:08.352216 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcf605-4a33-4711-9790-2ca8367cd819 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:08.415025 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30f4ed-9189-4b5b-8daa-f8dd7a7021b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:08.415645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30f4ed-9189-4b5b-8daa-f8dd7a7021b1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:08.415932 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30f4ed-9189-4b5b-8daa-f8dd7a7021b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:08.416206 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30f4ed-9189-4b5b-8daa-f8dd7a7021b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:08.416948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30f4ed-9189-4b5b-8daa-f8dd7a7021b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:08.417303 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30f4ed-9189-4b5b-8daa-f8dd7a7021b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:08.418718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30f4ed-9189-4b5b-8daa-f8dd7a7021b1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:08.433422 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 301/613] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:08.439850 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e473e-66d6-4e0a-b69c-188560d82764 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:08.583460 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e473e-66d6-4e0a-b69c-188560d82764 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:08.621973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657e10b-cf90-42d9-bae2-85d9ec2f5a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:08.622774 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657e10b-cf90-42d9-bae2-85d9ec2f5a91 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:08.623186 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657e10b-cf90-42d9-bae2-85d9ec2f5a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:08.623430 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657e10b-cf90-42d9-bae2-85d9ec2f5a91 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:08.632212 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657e10b-cf90-42d9-bae2-85d9ec2f5a91 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:08.633686 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657e10b-cf90-42d9-bae2-85d9ec2f5a91 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:08.649687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-5657e10b-cf90-42d9-bae2-85d9ec2f5a91 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 14:08:08.649687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:08.649687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:08.649687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:08.649687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:08.649687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:08.649687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:08.649687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:08.649687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:08.649687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:08.649687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:08.649687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:08.649687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:08:08.649687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:08:08.649687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:08:08.649687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:08:08.650694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:08.650694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:08.650694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:08:08.650694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:08.650694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:08:08.650694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:08.650694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:08.650694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:08.650694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:08.650694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:08.650694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:08.650694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:08.650694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:08.650694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:08.650694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:08.650694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:08.650694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:08.650694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:08.651657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:08.651657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:08.651657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:08:08.651657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:08:08.651657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:08:08.651657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:08:08.651657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:08:08.651657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:08:08.651657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 14:08:08.651657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:08.655550 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 313/614] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:08:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:08.657277 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30f4ed-9189-4b5b-8daa-f8dd7a7021b1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:08.717601 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:08.778060 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30f4ed-9189-4b5b-8daa-f8dd7a7021b1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:08.835188 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306e027-050f-495b-ab66-7f98bfe3e589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:08.836249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306e027-050f-495b-ab66-7f98bfe3e589 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:08.836687 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306e027-050f-495b-ab66-7f98bfe3e589 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:08.837114 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306e027-050f-495b-ab66-7f98bfe3e589 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:08.838562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3306e027-050f-495b-ab66-7f98bfe3e589 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:08.839049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3306e027-050f-495b-ab66-7f98bfe3e589 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:08.841554 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3306e027-050f-495b-ab66-7f98bfe3e589 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:08.869564 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 302/615] 10.222.0.81 () {56 vars in 1135 bytes} [Tue Jun 7 14:08:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:08.875293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657e10b-cf90-42d9-bae2-85d9ec2f5a91 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:08.919836 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:09.000008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657e10b-cf90-42d9-bae2-85d9ec2f5a91 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:09.047021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb8c368-20e6-4852-aa4c-c1829e820911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:09.047861 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8c368-20e6-4852-aa4c-c1829e820911 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:09.048139 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8c368-20e6-4852-aa4c-c1829e820911 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:09.048410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8c368-20e6-4852-aa4c-c1829e820911 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:09.062918 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb8c368-20e6-4852-aa4c-c1829e820911 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:09.064673 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb8c368-20e6-4852-aa4c-c1829e820911 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:09.093784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-2eb8c368-20e6-4852-aa4c-c1829e820911 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:09.093784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:09.093784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:09.093784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:09.093784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:09.093784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:09.093784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:09.093784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:09.093784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:09.093784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:09.093784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:09.093784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:09.093784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:09.093784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:09.093784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:09.094749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:09.094749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:09.094749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:09.094749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:09.094749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:09.094749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:09.094749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:09.094749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:09.094749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:09.094749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:09.094749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:09.094749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:09.094749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:09.094749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:09.094749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:09.094749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:09.094749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:09.096739 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:09.096739 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:09.096739 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:09.096739 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:09.096739 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 314/616] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:09.102316 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306e027-050f-495b-ab66-7f98bfe3e589 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:09.142909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79175ba9-07ff-4a00-8793-306d65d01a99 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:09.143268 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79175ba9-07ff-4a00-8793-306d65d01a99 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:09.143461 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79175ba9-07ff-4a00-8793-306d65d01a99 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:09.145072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 315/617] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:09.149748 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f04d61-a353-4866-a993-51ea41e136f6 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:09.150029 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f04d61-a353-4866-a993-51ea41e136f6 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:09.150302 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f04d61-a353-4866-a993-51ea41e136f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:09.178410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-32f04d61-a353-4866-a993-51ea41e136f6 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:09.180217 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-32f04d61-a353-4866-a993-51ea41e136f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:09.295792 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306e027-050f-495b-ab66-7f98bfe3e589 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:09.317219 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 316/618] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:08:09.325682 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:09.352534 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f04d61-a353-4866-a993-51ea41e136f6 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:09.363597 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d7186-a2fe-4d29-b9b0-6cd15081e206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:09.364561 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d7186-a2fe-4d29-b9b0-6cd15081e206 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:09.367265 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d7186-a2fe-4d29-b9b0-6cd15081e206 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:09.371705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d7186-a2fe-4d29-b9b0-6cd15081e206 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:09.373301 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d7186-a2fe-4d29-b9b0-6cd15081e206 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:09.374311 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:09.374614 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d7186-a2fe-4d29-b9b0-6cd15081e206 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:09.377275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d7186-a2fe-4d29-b9b0-6cd15081e206 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:09.401601 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 303/619] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:09.407012 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d7186-a2fe-4d29-b9b0-6cd15081e206 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:09.468082 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f04d61-a353-4866-a993-51ea41e136f6 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:09.511196 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4076200-0d7d-4e27-8ac8-c732d519edd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:09.512415 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4076200-0d7d-4e27-8ac8-c732d519edd9 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:09.513077 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4076200-0d7d-4e27-8ac8-c732d519edd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:09.514622 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4076200-0d7d-4e27-8ac8-c732d519edd9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:09.526235 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4076200-0d7d-4e27-8ac8-c732d519edd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:08:09.534487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-b4076200-0d7d-4e27-8ac8-c732d519edd9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:09.534487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:09.534487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:09.534487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:09.534487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:09.534487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:09.534487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:09.534487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:09.534487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:09.534487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:09.534487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:09.534487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:09.534487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:08:09.534487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:08:09.534487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:08:09.534487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:08:09.535455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:08:09.535455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:08:09.535455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:08:09.535455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:08:09.535455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:09.535455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:09.535455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:09.535455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:09.535455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:09.535455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:09.535455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:09.535455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:09.535455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:09.535455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:09.535455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:09.535455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:09.536299 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:09.536299 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:09.536299 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:09.536299 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:09.536299 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:08:09.536299 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:08:09.536299 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:08:09.536299 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:08:09.536299 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:08:09.536299 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:08:09.536299 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:09.536299 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:09.539661 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 317/620] 10.222.0.81 () {56 vars in 1102 bytes} [Tue Jun 7 14:08:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:09.541181 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a813b2c-0231-4def-9ee0-872dfa0a6e4e None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:09.541668 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a813b2c-0231-4def-9ee0-872dfa0a6e4e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:09.542142 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a813b2c-0231-4def-9ee0-872dfa0a6e4e None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:09.544059 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 318/621] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:09.545015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a813b2c-0231-4def-9ee0-872dfa0a6e4e None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:09.559630 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d7186-a2fe-4d29-b9b0-6cd15081e206 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:09.604070 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ddeed-ec3b-40ae-b10d-8e83485b336a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:09.605323 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ddeed-ec3b-40ae-b10d-8e83485b336a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:09.605615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ddeed-ec3b-40ae-b10d-8e83485b336a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:09.605891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ddeed-ec3b-40ae-b10d-8e83485b336a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:09.613880 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ddeed-ec3b-40ae-b10d-8e83485b336a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:09.615675 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ddeed-ec3b-40ae-b10d-8e83485b336a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:09.634593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-7b5ddeed-ec3b-40ae-b10d-8e83485b336a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:09.634593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:09.634593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:09.634593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:09.634593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:09.634593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:09.634593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:09.634593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:09.634593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:09.634593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:09.634593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:09.634593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:09.634593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:08:09.634593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:08:09.634593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:08:09.636082 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:08:09.636082 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:09.636082 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:09.636082 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:09.636082 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:09.636082 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:09.636082 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:09.636082 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:09.636082 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:09.636082 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:09.636082 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:09.636082 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:09.636082 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:09.636082 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:09.636082 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:09.636082 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:09.636082 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:09.637032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:08:09.637032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:08:09.637032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:08:09.637032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:08:09.637032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:08:09.637032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:08:09.637032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:09.637032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:09.638906 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 304/622] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:08:09.639695 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:09.641781 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce666e-808b-4c06-9961-bac61a34cb92 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:09.642362 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce666e-808b-4c06-9961-bac61a34cb92 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:09.642834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce666e-808b-4c06-9961-bac61a34cb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:09.662825 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7dce666e-808b-4c06-9961-bac61a34cb92 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:09.665144 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7dce666e-808b-4c06-9961-bac61a34cb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:09.670784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a813b2c-0231-4def-9ee0-872dfa0a6e4e None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:09.695446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:09.713144 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c2fcd-9895-40f2-b0c3-122020a0e2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:09.713898 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c2fcd-9895-40f2-b0c3-122020a0e2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:09.714251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c2fcd-9895-40f2-b0c3-122020a0e2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:09.714637 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c2fcd-9895-40f2-b0c3-122020a0e2a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:09.715489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c2fcd-9895-40f2-b0c3-122020a0e2a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:09.715867 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c2fcd-9895-40f2-b0c3-122020a0e2a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:09.717481 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c2fcd-9895-40f2-b0c3-122020a0e2a7 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:09.729103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 319/623] 10.222.0.81 () {56 vars in 1103 bytes} [Tue Jun 7 14:08:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:09.730240 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c2fcd-9895-40f2-b0c3-122020a0e2a7 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:09.747770 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 305/624] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:08:09.749001 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce666e-808b-4c06-9961-bac61a34cb92 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:09.835777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c2fcd-9895-40f2-b0c3-122020a0e2a7 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:09.861932 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce666e-808b-4c06-9961-bac61a34cb92 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:09.873826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1adb28-05ab-4245-837c-35bc0bb2656a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:09.874577 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1adb28-05ab-4245-837c-35bc0bb2656a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:09.874932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1adb28-05ab-4245-837c-35bc0bb2656a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:09.875324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1adb28-05ab-4245-837c-35bc0bb2656a None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:09.876186 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1adb28-05ab-4245-837c-35bc0bb2656a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:09.876561 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1adb28-05ab-4245-837c-35bc0bb2656a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:09.878058 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1adb28-05ab-4245-837c-35bc0bb2656a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:09.880916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:09.899359 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 320/625] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:08:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:09.899359 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1adb28-05ab-4245-837c-35bc0bb2656a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:09.910654 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670aead-74b5-4e31-abf2-dc5f1f99a391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:09.911445 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670aead-74b5-4e31-abf2-dc5f1f99a391 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:09.911845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670aead-74b5-4e31-abf2-dc5f1f99a391 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:09.912219 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670aead-74b5-4e31-abf2-dc5f1f99a391 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:09.924659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8670aead-74b5-4e31-abf2-dc5f1f99a391 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:09.926261 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8670aead-74b5-4e31-abf2-dc5f1f99a391 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:09.946398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-8670aead-74b5-4e31-abf2-dc5f1f99a391 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:09.946398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:09.946398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:09.946398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:09.946398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:09.946398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:09.946398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:09.946398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:09.946398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:09.946398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:09.946398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:09.946398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:09.946398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:09.946398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:09.946398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:09.947596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:09.947596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:09.947596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:09.947596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:09.947596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:09.947596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:09.947596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:09.947596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:09.947596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:09.947596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:09.947596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:09.947596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:09.947596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:09.947596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:09.947596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:09.947596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:09.947596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:09.948721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:09.948721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:09.948721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:09.948721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:09.950213 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 306/626] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:09.951139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670aead-74b5-4e31-abf2-dc5f1f99a391 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:09.968505 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:10.034552 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1adb28-05ab-4245-837c-35bc0bb2656a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:10.069928 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670aead-74b5-4e31-abf2-dc5f1f99a391 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:10.090373 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e971f55-64af-4a56-9791-697cf5ef5afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:10.092068 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e971f55-64af-4a56-9791-697cf5ef5afd None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:10.092576 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e971f55-64af-4a56-9791-697cf5ef5afd None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:10.093058 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e971f55-64af-4a56-9791-697cf5ef5afd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:10.110711 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e971f55-64af-4a56-9791-697cf5ef5afd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:08:10.111160 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b8732-707f-458e-b4a1-8d78bf776c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:10.112170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b8732-707f-458e-b4a1-8d78bf776c5d None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:10.112495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b8732-707f-458e-b4a1-8d78bf776c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:10.112767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b8732-707f-458e-b4a1-8d78bf776c5d None admin] PATH_INFO: `/v3/projects/f67bad575f8b4a47bee7c314497cb2fe/users/da7c0f99060a44eaab743ba71f159e88/roles/a72e5464b7b746978805326c224753bc` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:10.123100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-6e971f55-64af-4a56-9791-697cf5ef5afd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:10.123100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:10.123100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:10.123100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:10.123100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:10.123100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:10.123100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:10.123100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:10.123100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:10.123100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:10.123100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:10.123100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:10.123100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:08:10.123100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:08:10.123100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:08:10.123100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:08:10.124088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:08:10.124088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:08:10.124088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:08:10.124088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:08:10.124088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:10.124088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:10.124088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:10.124088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:10.124088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:10.124088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:10.124088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:10.124088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:10.124088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:10.124088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:10.124088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:10.124088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:10.124938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:10.124938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:10.124938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:10.124938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:10.124938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:08:10.124938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:08:10.124938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:08:10.124938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:08:10.124938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:08:10.124938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:08:10.124938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:10.124938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:10.128245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 321/627] 10.222.0.81 () {56 vars in 1102 bytes} [Tue Jun 7 14:08:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:10.129678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e971f55-64af-4a56-9791-697cf5ef5afd None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:10.141159 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49b8732-707f-458e-b4a1-8d78bf776c5d None admin] RBAC: Authorizing `identity:create_grant(project_id=f67bad575f8b4a47bee7c314497cb2fe, user_id=da7c0f99060a44eaab743ba71f159e88, role_id=a72e5464b7b746978805326c224753bc)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:10.143429 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49b8732-707f-458e-b4a1-8d78bf776c5d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:10.179266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 307/628] 10.222.0.81 () {58 vars in 1439 bytes} [Tue Jun 7 14:08:09 2022] PUT /identity/v3/projects/f67bad575f8b4a47bee7c314497cb2fe/users/da7c0f99060a44eaab743ba71f159e88/roles/a72e5464b7b746978805326c224753bc => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:08:10.179266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b8732-707f-458e-b4a1-8d78bf776c5d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:10.232560 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:10.310926 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:10.319893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e971f55-64af-4a56-9791-697cf5ef5afd None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:10.376478 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b8732-707f-458e-b4a1-8d78bf776c5d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:10.393233 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd998e74-c756-46c6-a1e4-b340deefe1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:10.393975 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd998e74-c756-46c6-a1e4-b340deefe1de None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:10.394262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd998e74-c756-46c6-a1e4-b340deefe1de None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:10.394538 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd998e74-c756-46c6-a1e4-b340deefe1de None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:10.395296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd998e74-c756-46c6-a1e4-b340deefe1de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:10.395606 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd998e74-c756-46c6-a1e4-b340deefe1de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:10.397612 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd998e74-c756-46c6-a1e4-b340deefe1de None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:10.424624 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 322/629] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:10.428546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd998e74-c756-46c6-a1e4-b340deefe1de None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:10.450683 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3422fa-4410-4669-9b56-b05b035d09f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:10.451416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3422fa-4410-4669-9b56-b05b035d09f5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:10.451701 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3422fa-4410-4669-9b56-b05b035d09f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:10.451970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3422fa-4410-4669-9b56-b05b035d09f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:10.452733 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3422fa-4410-4669-9b56-b05b035d09f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:10.453034 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3422fa-4410-4669-9b56-b05b035d09f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:10.454412 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3422fa-4410-4669-9b56-b05b035d09f5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:10.464824 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 308/630] 10.222.0.81 () {56 vars in 1103 bytes} [Tue Jun 7 14:08:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:10.466167 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43e50f-8d3c-4c04-8c1a-ccd0ed0d54e8 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:10.466509 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43e50f-8d3c-4c04-8c1a-ccd0ed0d54e8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:10.466784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43e50f-8d3c-4c04-8c1a-ccd0ed0d54e8 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:10.468291 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 309/631] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:10.471844 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43e50f-8d3c-4c04-8c1a-ccd0ed0d54e8 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:10.577769 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd998e74-c756-46c6-a1e4-b340deefe1de None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:10.584363 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43e50f-8d3c-4c04-8c1a-ccd0ed0d54e8 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:10.595030 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:10.615240 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e170ba4-f6e4-486f-a441-96df38223a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:10.616073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e170ba4-f6e4-486f-a441-96df38223a0e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:10.616355 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e170ba4-f6e4-486f-a441-96df38223a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:10.616634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e170ba4-f6e4-486f-a441-96df38223a0e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:10.618304 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:10.624603 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e170ba4-f6e4-486f-a441-96df38223a0e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:10.626052 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e170ba4-f6e4-486f-a441-96df38223a0e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:10.629255 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f264a60-1ba7-47bf-8000-15d25645c069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:10.630275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f264a60-1ba7-47bf-8000-15d25645c069 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:10.630745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f264a60-1ba7-47bf-8000-15d25645c069 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:10.631212 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f264a60-1ba7-47bf-8000-15d25645c069 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:10.641285 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-4e170ba4-f6e4-486f-a441-96df38223a0e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 07 14:08:10.641285 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:10.641285 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:10.641285 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:10.641285 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:10.641285 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:10.641285 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:10.641285 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:10.641285 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:10.641285 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:10.641285 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:10.641285 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:10.641285 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:08:10.641285 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:08:10.641285 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:08:10.641285 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:08:10.642305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:10.642305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:10.642305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:08:10.642305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:10.642305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:08:10.642305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:10.642305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:10.642305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:10.642305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:10.642305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:10.642305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:10.642305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:10.642305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:10.642305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:10.642305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:10.642305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:10.642305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:10.642305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:10.648815 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:10.648815 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:10.648815 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:08:10.648815 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:08:10.648815 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:08:10.648815 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:08:10.648815 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:08:10.648815 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:08:10.648815 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 07 14:08:10.648815 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:10.648815 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 323/632] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:08:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:08:10.649440 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f264a60-1ba7-47bf-8000-15d25645c069 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:10.649515 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3046b7-b040-4f48-9d32-565e9cefc65e None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:10.649515 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3046b7-b040-4f48-9d32-565e9cefc65e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:10.649515 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3046b7-b040-4f48-9d32-565e9cefc65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:10.649717 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f264a60-1ba7-47bf-8000-15d25645c069 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:10.663412 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-3f264a60-1ba7-47bf-8000-15d25645c069 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:10.663412 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:10.663412 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:10.663412 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:10.663412 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:10.663412 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:10.663412 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:10.663412 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:10.663412 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:10.663412 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:10.663412 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:10.663412 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:10.663412 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:10.663412 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:10.663412 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:10.664364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:10.664364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:10.664364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:10.664364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:10.664364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:10.664364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:10.664364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:10.664364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:10.664364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:10.664364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:10.664364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:10.664364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:10.664364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:10.664364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:10.664364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:10.664364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:10.664364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:10.665315 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:10.665315 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:10.665315 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:10.665315 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:10.665315 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 310/633] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:10.666262 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3de7f-3211-4867-ac88-fce2963d4454 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:10.666444 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3de7f-3211-4867-ac88-fce2963d4454 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:10.666688 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3de7f-3211-4867-ac88-fce2963d4454 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:10.671268 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 311/634] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:10.671268 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3de7f-3211-4867-ac88-fce2963d4454 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:10.671477 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-de3046b7-b040-4f48-9d32-565e9cefc65e None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:10.672751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-de3046b7-b040-4f48-9d32-565e9cefc65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:10.744883 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 324/635] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:08:10.746293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3046b7-b040-4f48-9d32-565e9cefc65e None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:10.789303 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3de7f-3211-4867-ac88-fce2963d4454 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:10.826378 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116fc11-dafe-444b-a0e2-73fa151c2b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:10.826770 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116fc11-dafe-444b-a0e2-73fa151c2b2f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:10.827123 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116fc11-dafe-444b-a0e2-73fa151c2b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:10.827498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116fc11-dafe-444b-a0e2-73fa151c2b2f None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:10.828409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7116fc11-dafe-444b-a0e2-73fa151c2b2f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:10.828788 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7116fc11-dafe-444b-a0e2-73fa151c2b2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:10.830285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7116fc11-dafe-444b-a0e2-73fa151c2b2f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:10.849373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 312/636] 10.222.0.81 () {56 vars in 1137 bytes} [Tue Jun 7 14:08:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:10.850824 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f337e3-a90d-4d96-a01f-a5c8cbc0bdbe None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:10.851107 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f337e3-a90d-4d96-a01f-a5c8cbc0bdbe None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:10.851571 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f337e3-a90d-4d96-a01f-a5c8cbc0bdbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:10.864944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3046b7-b040-4f48-9d32-565e9cefc65e None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:10.867905 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f4f337e3-a90d-4d96-a01f-a5c8cbc0bdbe None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:10.869096 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f4f337e3-a90d-4d96-a01f-a5c8cbc0bdbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:10.874401 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:10.902477 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:10.904631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd15a7-3810-454f-99eb-9de7ac5d711e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:10.905545 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd15a7-3810-454f-99eb-9de7ac5d711e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:10.905827 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd15a7-3810-454f-99eb-9de7ac5d711e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:10.906097 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd15a7-3810-454f-99eb-9de7ac5d711e None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:10.906819 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd15a7-3810-454f-99eb-9de7ac5d711e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:10.907122 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd15a7-3810-454f-99eb-9de7ac5d711e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:10.908517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd15a7-3810-454f-99eb-9de7ac5d711e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:10.922151 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 325/637] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:10.923152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd15a7-3810-454f-99eb-9de7ac5d711e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:10.950263 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 313/638] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:08:10.951261 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f337e3-a90d-4d96-a01f-a5c8cbc0bdbe None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:11.039716 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd15a7-3810-454f-99eb-9de7ac5d711e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:11.063683 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f337e3-a90d-4d96-a01f-a5c8cbc0bdbe None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:11.073085 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d58c7-1e18-4f13-8d12-3ef4cda27c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:11.073980 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d58c7-1e18-4f13-8d12-3ef4cda27c46 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:11.074369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d58c7-1e18-4f13-8d12-3ef4cda27c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:11.074732 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d58c7-1e18-4f13-8d12-3ef4cda27c46 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:11.085626 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d58c7-1e18-4f13-8d12-3ef4cda27c46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:08:11.093483 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-2e3d58c7-1e18-4f13-8d12-3ef4cda27c46 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:11.093483 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:11.093483 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:11.093483 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:11.093483 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:11.093483 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:11.093483 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:11.093483 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:11.093483 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:11.093483 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:11.093483 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:11.093483 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:11.093483 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:08:11.093483 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:08:11.093483 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:08:11.093483 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:08:11.094631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:08:11.094631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:08:11.094631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:08:11.094631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:08:11.094631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:11.094631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:11.094631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:11.094631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:11.094631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:11.094631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:11.094631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:11.094631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:11.094631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:11.094631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:11.094631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:11.094631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:11.095718 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:11.095718 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:11.095718 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:11.095718 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:11.095718 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:08:11.095718 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:08:11.095718 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:08:11.095718 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:08:11.095718 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:08:11.095718 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:08:11.095718 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:11.095718 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:11.102088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 326/639] 10.222.0.81 () {56 vars in 1102 bytes} [Tue Jun 7 14:08:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:11.102088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d58c7-1e18-4f13-8d12-3ef4cda27c46 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:11.107466 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16cb965-a262-446a-9b75-c81ed4782a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:11.108206 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16cb965-a262-446a-9b75-c81ed4782a4c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:11.108567 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16cb965-a262-446a-9b75-c81ed4782a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:11.108910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16cb965-a262-446a-9b75-c81ed4782a4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:11.129926 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:11.143715 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:11.182205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16cb965-a262-446a-9b75-c81ed4782a4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:11.183695 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16cb965-a262-446a-9b75-c81ed4782a4c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:11.214876 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d58c7-1e18-4f13-8d12-3ef4cda27c46 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:11.249419 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0416d-992b-4a7a-a736-91b7a75941da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:11.250209 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0416d-992b-4a7a-a736-91b7a75941da None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:11.250489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0416d-992b-4a7a-a736-91b7a75941da None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:11.250765 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0416d-992b-4a7a-a736-91b7a75941da None admin] PATH_INFO: `/v3/users/nova` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:11.260018 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff0416d-992b-4a7a-a736-91b7a75941da None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:11.261861 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff0416d-992b-4a7a-a736-91b7a75941da None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:11.281021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-eff0416d-992b-4a7a-a736-91b7a75941da None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:08:11.281021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:11.281021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:11.281021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:11.281021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:11.281021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:11.281021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:11.281021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:11.281021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:11.281021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:11.281021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:11.281021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:11.281021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:08:11.281021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:08:11.281021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:08:11.281021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:08:11.282021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:11.282021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:11.282021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:08:11.282021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:11.282021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:08:11.282021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:11.282021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:11.282021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:11.282021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:11.282021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:11.282021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:11.282021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:11.282021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:11.282021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:11.282021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:11.282021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:11.282021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:11.282021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:11.282981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:11.282981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:11.282981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:08:11.282981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:08:11.282981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:08:11.282981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:08:11.282981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:08:11.282981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:08:11.282981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:08:11.282981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:11.284101 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 327/640] 10.222.0.81 () {56 vars in 1093 bytes} [Tue Jun 7 14:08:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:11.284179 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0416d-992b-4a7a-a736-91b7a75941da None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:11.290926 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 314/641] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:08:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:08:11.291800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16cb965-a262-446a-9b75-c81ed4782a4c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:11.362235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:11.377626 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:11.393923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0416d-992b-4a7a-a736-91b7a75941da None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:11.402178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16cb965-a262-446a-9b75-c81ed4782a4c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:11.431427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c459af-1880-4fdc-b127-7d99ebaf3af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:11.432144 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c459af-1880-4fdc-b127-7d99ebaf3af9 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:11.432427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c459af-1880-4fdc-b127-7d99ebaf3af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:11.432707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c459af-1880-4fdc-b127-7d99ebaf3af9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:11.440366 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3922bac2-c5e8-449f-b5cb-e54b5040584d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:11.441034 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922bac2-c5e8-449f-b5cb-e54b5040584d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:11.441322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922bac2-c5e8-449f-b5cb-e54b5040584d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:11.441598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922bac2-c5e8-449f-b5cb-e54b5040584d None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:11.442345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3922bac2-c5e8-449f-b5cb-e54b5040584d None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:11.442655 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3922bac2-c5e8-449f-b5cb-e54b5040584d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:11.443382 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c459af-1880-4fdc-b127-7d99ebaf3af9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:08:11.444190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3922bac2-c5e8-449f-b5cb-e54b5040584d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:11.451293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-91c459af-1880-4fdc-b127-7d99ebaf3af9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:11.451293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:11.451293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:11.451293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:11.451293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:11.451293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:11.451293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:11.451293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:11.451293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:11.451293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:11.451293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:11.451293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:11.451293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:08:11.451293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:08:11.451293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:08:11.451293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:08:11.452309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:08:11.452309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:08:11.452309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:08:11.452309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:08:11.452309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:11.452309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:11.452309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:11.452309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:11.452309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:11.452309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:11.452309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:11.452309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:11.452309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:11.452309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:11.452309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:11.452309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:11.460561 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:11.460561 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:11.460561 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:11.460561 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:11.460561 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:08:11.460561 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:08:11.460561 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:08:11.460561 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:08:11.460561 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:08:11.460561 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:08:11.460561 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:11.460561 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:11.461360 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 315/643] 10.222.0.81 () {56 vars in 1103 bytes} [Tue Jun 7 14:08:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:11.461447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 328/643] 10.222.0.81 () {56 vars in 1102 bytes} [Tue Jun 7 14:08:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:11.461447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c459af-1880-4fdc-b127-7d99ebaf3af9 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:11.461601 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3922bac2-c5e8-449f-b5cb-e54b5040584d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:11.566474 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c459af-1880-4fdc-b127-7d99ebaf3af9 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:11.572189 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3922bac2-c5e8-449f-b5cb-e54b5040584d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:11.600920 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:11.603083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7106d-318e-448d-b527-3b3c8d20cdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:11.604119 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7106d-318e-448d-b527-3b3c8d20cdf2 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:11.604500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7106d-318e-448d-b527-3b3c8d20cdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:11.604870 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7106d-318e-448d-b527-3b3c8d20cdf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:11.611007 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c916208-71b7-42df-85f4-c3c5b88188c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:11.612174 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c916208-71b7-42df-85f4-c3c5b88188c9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:11.612467 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c916208-71b7-42df-85f4-c3c5b88188c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:11.612747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c916208-71b7-42df-85f4-c3c5b88188c9 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:11.613592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c916208-71b7-42df-85f4-c3c5b88188c9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:11.613909 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c916208-71b7-42df-85f4-c3c5b88188c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:11.615827 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c916208-71b7-42df-85f4-c3c5b88188c9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:11.616733 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7106d-318e-448d-b527-3b3c8d20cdf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:11.618209 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7106d-318e-448d-b527-3b3c8d20cdf2 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:11.625407 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:11.638798 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 316/644] 10.222.0.81 () {56 vars in 1133 bytes} [Tue Jun 7 14:08:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:11.638798 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c916208-71b7-42df-85f4-c3c5b88188c9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:11.639019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-27f7106d-318e-448d-b527-3b3c8d20cdf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:11.639019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:11.639019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:11.639019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:11.639019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:11.639019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:11.639019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:11.639019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:11.639019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:11.639019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:11.639019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:11.639019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:11.639019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:11.639019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:11.639019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:11.639927 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:11.639927 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:11.639927 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:11.639927 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:11.639927 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:11.639927 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:11.639927 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:11.639927 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:11.639927 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:11.639927 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:11.639927 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:11.639927 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:11.639927 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:11.639927 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:11.639927 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:11.639927 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:11.639927 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:11.640978 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:11.640978 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:11.640978 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:11.640978 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:11.640978 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 329/645] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:11.640978 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7106d-318e-448d-b527-3b3c8d20cdf2 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:11.748802 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7106d-318e-448d-b527-3b3c8d20cdf2 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:11.749244 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c916208-71b7-42df-85f4-c3c5b88188c9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:11.785060 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b85f7b-1023-40d6-b95e-49ef69b97e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:11.786014 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b85f7b-1023-40d6-b95e-49ef69b97e1e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:11.786478 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b85f7b-1023-40d6-b95e-49ef69b97e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:11.786828 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b85f7b-1023-40d6-b95e-49ef69b97e1e None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:11.787272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70160230-839d-48fa-bc89-0eafa2ffa82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:11.787623 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b85f7b-1023-40d6-b95e-49ef69b97e1e None admin] RBAC: Adding query filter params (name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:11.788455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b85f7b-1023-40d6-b95e-49ef69b97e1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:11.788759 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70160230-839d-48fa-bc89-0eafa2ffa82c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:11.789046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70160230-839d-48fa-bc89-0eafa2ffa82c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:11.789319 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70160230-839d-48fa-bc89-0eafa2ffa82c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:11.789869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b85f7b-1023-40d6-b95e-49ef69b97e1e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:11.798105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 330/646] 10.222.0.81 () {56 vars in 1103 bytes} [Tue Jun 7 14:08:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:11.798105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b85f7b-1023-40d6-b95e-49ef69b97e1e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:11.803908 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70160230-839d-48fa-bc89-0eafa2ffa82c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:11.805277 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70160230-839d-48fa-bc89-0eafa2ffa82c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:11.823467 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-70160230-839d-48fa-bc89-0eafa2ffa82c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:11.823467 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:11.823467 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:11.823467 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:11.823467 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:11.823467 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:11.823467 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:11.823467 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:11.823467 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:11.823467 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:11.823467 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:11.823467 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:11.823467 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:11.823467 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:11.823467 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:11.824385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:11.824385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:11.824385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:11.824385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:11.824385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:11.824385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:11.824385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:11.824385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:11.824385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:11.824385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:11.824385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:11.824385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:11.824385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:11.824385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:11.824385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:11.824385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:11.824385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:11.825265 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:11.825265 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:11.825265 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:11.825265 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:11.825265 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 317/647] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:11.825548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70160230-839d-48fa-bc89-0eafa2ffa82c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:11.867882 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:11.873133 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:11.911697 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b85f7b-1023-40d6-b95e-49ef69b97e1e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:11.945261 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70160230-839d-48fa-bc89-0eafa2ffa82c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:11.948370 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d275ef0-9b27-4a95-8954-73ec96e5eeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:11.949026 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d275ef0-9b27-4a95-8954-73ec96e5eeb4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:11.949305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d275ef0-9b27-4a95-8954-73ec96e5eeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:11.949575 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d275ef0-9b27-4a95-8954-73ec96e5eeb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:11.985904 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f420a8-711a-45f0-bd42-965374dbf8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:11.986575 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f420a8-711a-45f0-bd42-965374dbf8b6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:11.986853 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f420a8-711a-45f0-bd42-965374dbf8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:11.987130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f420a8-711a-45f0-bd42-965374dbf8b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:11.987876 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f420a8-711a-45f0-bd42-965374dbf8b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:11.988178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f420a8-711a-45f0-bd42-965374dbf8b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:11.989535 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f420a8-711a-45f0-bd42-965374dbf8b6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:12.006110 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 318/648] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:12.006110 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f420a8-711a-45f0-bd42-965374dbf8b6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:12.028025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d275ef0-9b27-4a95-8954-73ec96e5eeb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:12.029487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d275ef0-9b27-4a95-8954-73ec96e5eeb4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:12.093371 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:12.117217 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:12.121425 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f420a8-711a-45f0-bd42-965374dbf8b6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:12.140028 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 331/649] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:08:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:08:12.141082 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d275ef0-9b27-4a95-8954-73ec96e5eeb4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:12.161418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65cbe9-1875-4775-babf-4a640da02e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:12.162474 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65cbe9-1875-4775-babf-4a640da02e0a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:12.163552 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65cbe9-1875-4775-babf-4a640da02e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:12.163902 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65cbe9-1875-4775-babf-4a640da02e0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:12.176113 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65cbe9-1875-4775-babf-4a640da02e0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:12.177541 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65cbe9-1875-4775-babf-4a640da02e0a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:12.195482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-bd65cbe9-1875-4775-babf-4a640da02e0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:12.195482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:12.195482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:12.195482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:12.195482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:12.195482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:12.195482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:12.195482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:12.195482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:12.195482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:12.195482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:12.195482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:12.195482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:12.195482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:12.195482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:12.196495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:12.196495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:12.196495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:12.196495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:12.196495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:12.196495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:12.196495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:12.196495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:12.196495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:12.196495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:12.196495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:12.196495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:12.196495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:12.196495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:12.196495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:12.196495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:12.196495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:12.197376 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:12.197376 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:12.197376 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:12.197376 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:12.197376 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 319/650] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:12.198090 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65cbe9-1875-4775-babf-4a640da02e0a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:12.259876 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d275ef0-9b27-4a95-8954-73ec96e5eeb4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:12.296356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f3af47-d434-4d27-b6af-4c9a636690f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:12.297181 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3af47-d434-4d27-b6af-4c9a636690f3 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:12.297292 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3af47-d434-4d27-b6af-4c9a636690f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:12.297577 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3af47-d434-4d27-b6af-4c9a636690f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:12.298301 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f3af47-d434-4d27-b6af-4c9a636690f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:12.298610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f3af47-d434-4d27-b6af-4c9a636690f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:12.300004 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f3af47-d434-4d27-b6af-4c9a636690f3 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:12.313026 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 332/651] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:12.313673 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f3af47-d434-4d27-b6af-4c9a636690f3 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:12.317851 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65cbe9-1875-4775-babf-4a640da02e0a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:12.336492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:12.354075 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1db98b-935c-4168-a411-850890e68b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:12.354879 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1db98b-935c-4168-a411-850890e68b8b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:12.355161 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1db98b-935c-4168-a411-850890e68b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:12.355465 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1db98b-935c-4168-a411-850890e68b8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:12.362437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1db98b-935c-4168-a411-850890e68b8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:12.363833 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1db98b-935c-4168-a411-850890e68b8b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:12.379209 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-bf1db98b-935c-4168-a411-850890e68b8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:12.379209 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:12.379209 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:12.379209 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:12.379209 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:12.379209 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:12.379209 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:12.379209 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:12.379209 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:12.379209 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:12.379209 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:12.379209 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:12.379209 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:08:12.379209 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:08:12.379209 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:08:12.380148 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:08:12.380148 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:12.380148 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:12.380148 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:12.380148 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:12.380148 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:12.380148 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:12.380148 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:12.380148 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:12.380148 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:12.380148 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:12.380148 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:12.380148 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:12.380148 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:12.380148 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:12.380148 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:12.380148 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:12.381049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:08:12.381049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:08:12.381049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:08:12.381049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:08:12.381049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:08:12.381049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:08:12.381049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:12.381049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:12.381049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 320/652] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:12.381591 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1db98b-935c-4168-a411-850890e68b8b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:12.397480 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:12.430540 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f3af47-d434-4d27-b6af-4c9a636690f3 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:12.468249 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a30e2-2b76-48b2-91c4-f5f63076abbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:12.469035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a30e2-2b76-48b2-91c4-f5f63076abbe None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:12.469318 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a30e2-2b76-48b2-91c4-f5f63076abbe None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:12.469594 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a30e2-2b76-48b2-91c4-f5f63076abbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:12.482554 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a30e2-2b76-48b2-91c4-f5f63076abbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:12.484077 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a30e2-2b76-48b2-91c4-f5f63076abbe None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:12.497000 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1db98b-935c-4168-a411-850890e68b8b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:12.502914 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-174a30e2-2b76-48b2-91c4-f5f63076abbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:12.502914 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:12.502914 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:12.502914 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:12.502914 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:12.502914 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:12.502914 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:12.502914 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:12.502914 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:12.502914 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:12.502914 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:12.502914 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:12.502914 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:12.502914 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:12.502914 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:12.504073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:12.504073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:12.504073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:12.504073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:12.504073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:12.504073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:12.504073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:12.504073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:12.504073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:12.504073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:12.504073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:12.504073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:12.504073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:12.504073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:12.504073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:12.504073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:12.504073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:12.505169 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:12.505169 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:12.505169 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:12.505169 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:12.505169 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 333/653] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:12.505478 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a30e2-2b76-48b2-91c4-f5f63076abbe None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:12.541393 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205209c9-16c1-40e2-9cc4-cd7b06fd11ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:12.541983 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205209c9-16c1-40e2-9cc4-cd7b06fd11ee None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:12.542277 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205209c9-16c1-40e2-9cc4-cd7b06fd11ee None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:12.542554 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205209c9-16c1-40e2-9cc4-cd7b06fd11ee None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:12.543327 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205209c9-16c1-40e2-9cc4-cd7b06fd11ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:12.543637 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205209c9-16c1-40e2-9cc4-cd7b06fd11ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:12.545010 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205209c9-16c1-40e2-9cc4-cd7b06fd11ee None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:12.557718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 321/654] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:12.562862 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205209c9-16c1-40e2-9cc4-cd7b06fd11ee None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:12.580231 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:12.610206 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a30e2-2b76-48b2-91c4-f5f63076abbe None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:12.632719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:12.645807 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cae2b1-22e5-4bcd-81ac-b0436da0d7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:12.646711 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cae2b1-22e5-4bcd-81ac-b0436da0d7da None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:12.647183 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cae2b1-22e5-4bcd-81ac-b0436da0d7da None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:12.647493 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cae2b1-22e5-4bcd-81ac-b0436da0d7da None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:12.655678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cae2b1-22e5-4bcd-81ac-b0436da0d7da None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:12.657121 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cae2b1-22e5-4bcd-81ac-b0436da0d7da None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:12.663794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205209c9-16c1-40e2-9cc4-cd7b06fd11ee None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:12.671083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-78cae2b1-22e5-4bcd-81ac-b0436da0d7da None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 14:08:12.671083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:12.671083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:12.671083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:12.671083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:12.671083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:12.671083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:12.671083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:12.671083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:12.671083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:12.671083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:12.671083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:12.671083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:08:12.671083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:08:12.671083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:08:12.671083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:08:12.672046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:12.672046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:12.672046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:08:12.672046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:12.672046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:08:12.672046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:12.672046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:12.672046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:12.672046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:12.672046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:12.672046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:12.672046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:12.672046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:12.672046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:12.672046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:12.672046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:12.672046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:12.672046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:12.673004 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:12.673004 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:12.673004 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:08:12.673004 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:08:12.673004 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:08:12.673004 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:08:12.673004 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:08:12.673004 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:08:12.673004 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 14:08:12.673004 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:12.675530 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 334/655] 10.222.0.81 () {56 vars in 1102 bytes} [Tue Jun 7 14:08:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:12.676634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cae2b1-22e5-4bcd-81ac-b0436da0d7da None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:12.703096 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659690d-ba77-48a6-b9a8-d23db5176e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:12.703794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659690d-ba77-48a6-b9a8-d23db5176e19 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:12.704078 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659690d-ba77-48a6-b9a8-d23db5176e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:12.704353 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659690d-ba77-48a6-b9a8-d23db5176e19 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:12.705141 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9659690d-ba77-48a6-b9a8-d23db5176e19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:12.705451 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9659690d-ba77-48a6-b9a8-d23db5176e19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:12.706871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9659690d-ba77-48a6-b9a8-d23db5176e19 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:12.718771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 322/656] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:08:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:12.719588 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659690d-ba77-48a6-b9a8-d23db5176e19 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:12.776499 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cae2b1-22e5-4bcd-81ac-b0436da0d7da None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:12.804988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:12.810609 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088cf9be-5630-4976-8972-6cddd977c8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:12.811289 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cf9be-5630-4976-8972-6cddd977c8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:12.811573 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cf9be-5630-4976-8972-6cddd977c8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:12.811852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cf9be-5630-4976-8972-6cddd977c8d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:12.812574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088cf9be-5630-4976-8972-6cddd977c8d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:12.812878 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088cf9be-5630-4976-8972-6cddd977c8d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:12.814230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088cf9be-5630-4976-8972-6cddd977c8d3 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:12.819524 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659690d-ba77-48a6-b9a8-d23db5176e19 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:12.825869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 335/657] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:12.827109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765824a-6d01-47bc-8dde-5b7341c372bc None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:12.827425 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765824a-6d01-47bc-8dde-5b7341c372bc None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:12.827701 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765824a-6d01-47bc-8dde-5b7341c372bc None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:12.828911 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 336/658] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:12.829598 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765824a-6d01-47bc-8dde-5b7341c372bc None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:12.853128 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:12.858985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f371c-d0f5-4627-a091-df47065350d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:12.859806 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f371c-d0f5-4627-a091-df47065350d0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:12.860089 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f371c-d0f5-4627-a091-df47065350d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:12.860364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f371c-d0f5-4627-a091-df47065350d0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:12.867478 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8f371c-d0f5-4627-a091-df47065350d0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:12.868917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8f371c-d0f5-4627-a091-df47065350d0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:12.883990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-7a8f371c-d0f5-4627-a091-df47065350d0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 14:08:12.883990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:12.883990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:12.883990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:12.883990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:12.883990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:12.883990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:12.883990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:12.883990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:12.883990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:12.883990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:12.883990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:12.883990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:08:12.883990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:08:12.883990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:08:12.883990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:08:12.885119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:12.885119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:12.885119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:08:12.885119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:12.885119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:08:12.885119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:12.885119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:12.885119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:12.885119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:12.885119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:12.885119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:12.885119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:12.885119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:12.885119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:12.885119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:12.885119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:12.885119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:12.885119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:12.886078 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:12.886078 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:12.886078 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:08:12.886078 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:08:12.886078 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:08:12.886078 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:08:12.886078 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:08:12.886078 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:08:12.886078 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 14:08:12.886078 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:12.886078 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 323/659] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:08:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:12.886078 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f371c-d0f5-4627-a091-df47065350d0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:12.934723 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765824a-6d01-47bc-8dde-5b7341c372bc None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:12.969466 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc729ec9-8959-4363-a6c9-318e7540cf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:12.970131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc729ec9-8959-4363-a6c9-318e7540cf3f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:12.970411 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc729ec9-8959-4363-a6c9-318e7540cf3f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:12.970682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc729ec9-8959-4363-a6c9-318e7540cf3f None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:12.971545 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc729ec9-8959-4363-a6c9-318e7540cf3f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:12.971853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc729ec9-8959-4363-a6c9-318e7540cf3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:12.973265 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc729ec9-8959-4363-a6c9-318e7540cf3f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:12.989949 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 337/660] 10.222.0.81 () {56 vars in 1139 bytes} [Tue Jun 7 14:08:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:12.990798 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc729ec9-8959-4363-a6c9-318e7540cf3f None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:12.990906 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f371c-d0f5-4627-a091-df47065350d0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:13.026316 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940cc7ae-1830-49d4-a66b-bf32ee56b4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:13.026970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940cc7ae-1830-49d4-a66b-bf32ee56b4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:13.027348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940cc7ae-1830-49d4-a66b-bf32ee56b4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:13.027628 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940cc7ae-1830-49d4-a66b-bf32ee56b4b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:13.028458 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940cc7ae-1830-49d4-a66b-bf32ee56b4b3 None admin] RBAC: Adding query filter params (role.id=a72e5464b7b746978805326c224753bc, scope.project.id=f67bad575f8b4a47bee7c314497cb2fe, user.id=b6180393da0f495db964f0b2588d8b90) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:13.028767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940cc7ae-1830-49d4-a66b-bf32ee56b4b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:13.030196 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940cc7ae-1830-49d4-a66b-bf32ee56b4b3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:13.036036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 324/661] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:08:12 2022] GET /identity/v3/role_assignments?user.id=b6180393da0f495db964f0b2588d8b90&scope.project.id=f67bad575f8b4a47bee7c314497cb2fe&role.id=a72e5464b7b746978805326c224753bc => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:13.037528 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c2677-1e68-40d3-9eea-4dd82dac9185 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:13.037875 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c2677-1e68-40d3-9eea-4dd82dac9185 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:13.038204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c2677-1e68-40d3-9eea-4dd82dac9185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:13.043189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:13.056137 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b95c2677-1e68-40d3-9eea-4dd82dac9185 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:13.057296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b95c2677-1e68-40d3-9eea-4dd82dac9185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:13.106408 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc729ec9-8959-4363-a6c9-318e7540cf3f None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:13.118564 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:13.139824 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 325/662] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:08:13.140631 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c2677-1e68-40d3-9eea-4dd82dac9185 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:13.175549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1c0a2-c9c6-4102-845a-ef6e271f8ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:13.176369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1c0a2-c9c6-4102-845a-ef6e271f8ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:13.176647 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1c0a2-c9c6-4102-845a-ef6e271f8ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:13.176922 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1c0a2-c9c6-4102-845a-ef6e271f8ca2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:13.185411 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f1c0a2-c9c6-4102-845a-ef6e271f8ca2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:13.186887 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f1c0a2-c9c6-4102-845a-ef6e271f8ca2 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:13.201912 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-10f1c0a2-c9c6-4102-845a-ef6e271f8ca2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:13.201912 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:13.201912 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:13.201912 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:13.201912 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:13.201912 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:13.201912 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:13.201912 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:13.201912 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:13.201912 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:13.201912 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:13.201912 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:13.201912 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:08:13.201912 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:08:13.201912 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:08:13.202904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:08:13.202904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:13.202904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:13.202904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:13.202904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:13.202904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:13.202904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:13.202904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:13.202904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:13.202904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:13.202904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:13.202904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:13.202904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:13.202904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:13.202904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:13.202904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:13.202904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:13.203948 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:08:13.203948 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:08:13.203948 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:08:13.203948 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:08:13.203948 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:08:13.203948 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:08:13.203948 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:13.203948 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:13.203948 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 338/663] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:13.204446 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1c0a2-c9c6-4102-845a-ef6e271f8ca2 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:13.271590 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c2677-1e68-40d3-9eea-4dd82dac9185 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:13.312203 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1659f39-1e34-40a6-9d5a-9280925f7d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:13.313068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1659f39-1e34-40a6-9d5a-9280925f7d49 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:13.313068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1659f39-1e34-40a6-9d5a-9280925f7d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:13.313355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1659f39-1e34-40a6-9d5a-9280925f7d49 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:13.314205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1659f39-1e34-40a6-9d5a-9280925f7d49 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:13.314533 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1659f39-1e34-40a6-9d5a-9280925f7d49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:13.315995 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1659f39-1e34-40a6-9d5a-9280925f7d49 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:13.317947 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1c0a2-c9c6-4102-845a-ef6e271f8ca2 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:13.328794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:13.339278 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 326/664] 10.222.0.81 () {56 vars in 1137 bytes} [Tue Jun 7 14:08:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:13.339278 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1659f39-1e34-40a6-9d5a-9280925f7d49 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:13.354774 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c77c1-5dba-4e69-9379-82fdecaaff9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:13.355573 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c77c1-5dba-4e69-9379-82fdecaaff9f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:13.355856 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c77c1-5dba-4e69-9379-82fdecaaff9f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:13.356130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c77c1-5dba-4e69-9379-82fdecaaff9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:13.383476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:13.430866 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c77c1-5dba-4e69-9379-82fdecaaff9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:13.432343 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c77c1-5dba-4e69-9379-82fdecaaff9f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:13.445955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1659f39-1e34-40a6-9d5a-9280925f7d49 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:13.481738 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ab130-78b4-4dee-a1b4-382f6aa90a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:13.482786 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ab130-78b4-4dee-a1b4-382f6aa90a84 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:13.483283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ab130-78b4-4dee-a1b4-382f6aa90a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:13.483562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ab130-78b4-4dee-a1b4-382f6aa90a84 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:13.495207 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ab130-78b4-4dee-a1b4-382f6aa90a84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:08:13.503804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-a63ab130-78b4-4dee-a1b4-382f6aa90a84 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:13.503804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:13.503804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:13.503804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:13.503804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:13.503804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:13.503804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:13.503804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:13.503804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:13.503804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:13.503804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:13.503804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:13.503804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:08:13.503804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:08:13.503804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:08:13.503804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:08:13.504849 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:08:13.504849 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:08:13.504849 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:08:13.504849 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:08:13.504849 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:13.504849 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:13.504849 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:13.504849 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:13.504849 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:13.504849 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:13.504849 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:13.504849 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:13.504849 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:13.504849 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:13.504849 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:13.504849 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:13.505691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:13.505691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:13.505691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:13.505691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:13.505691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:08:13.505691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:08:13.505691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:08:13.505691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:08:13.505691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:08:13.505691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:08:13.505691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:13.505691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:13.505691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 327/665] 10.222.0.81 () {56 vars in 1102 bytes} [Tue Jun 7 14:08:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:13.509492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ab130-78b4-4dee-a1b4-382f6aa90a84 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:13.542839 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 339/666] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:08:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:08:13.543708 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c77c1-5dba-4e69-9379-82fdecaaff9f None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:13.556534 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:13.621464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ab130-78b4-4dee-a1b4-382f6aa90a84 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:13.623095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:13.660341 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7404fa-b567-457f-a66a-41c8a6e1e4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:13.661001 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7404fa-b567-457f-a66a-41c8a6e1e4f3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:13.661282 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7404fa-b567-457f-a66a-41c8a6e1e4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:13.661555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7404fa-b567-457f-a66a-41c8a6e1e4f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:13.662356 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7404fa-b567-457f-a66a-41c8a6e1e4f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:13.663285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7404fa-b567-457f-a66a-41c8a6e1e4f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:13.663409 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c77c1-5dba-4e69-9379-82fdecaaff9f None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:13.664921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7404fa-b567-457f-a66a-41c8a6e1e4f3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:13.677525 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 328/667] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:08:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:13.679031 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7404fa-b567-457f-a66a-41c8a6e1e4f3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:13.707201 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5ee8c-be42-4c79-a858-c727bc640cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:13.707870 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5ee8c-be42-4c79-a858-c727bc640cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:13.708156 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5ee8c-be42-4c79-a858-c727bc640cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:13.708427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5ee8c-be42-4c79-a858-c727bc640cf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:13.778651 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5ee8c-be42-4c79-a858-c727bc640cf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:13.780140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5ee8c-be42-4c79-a858-c727bc640cf3 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:13.786874 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7404fa-b567-457f-a66a-41c8a6e1e4f3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:13.792663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:13.825716 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ffbc7-a813-4cfd-a230-eb4298cacce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:13.826706 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ffbc7-a813-4cfd-a230-eb4298cacce5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:13.827151 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ffbc7-a813-4cfd-a230-eb4298cacce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:13.827455 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ffbc7-a813-4cfd-a230-eb4298cacce5 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:13.828207 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ffbc7-a813-4cfd-a230-eb4298cacce5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:13.828515 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ffbc7-a813-4cfd-a230-eb4298cacce5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:13.829871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ffbc7-a813-4cfd-a230-eb4298cacce5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:13.838649 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 329/668] 10.222.0.81 () {56 vars in 1103 bytes} [Tue Jun 7 14:08:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:13.839492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ffbc7-a813-4cfd-a230-eb4298cacce5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:13.867243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:13.895023 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 340/669] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:08:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:08:13.896201 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5ee8c-be42-4c79-a858-c727bc640cf3 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:13.950718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ffbc7-a813-4cfd-a230-eb4298cacce5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:13.985674 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebfc0f4-97ac-4f62-8f75-855160a20584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:13.986484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfc0f4-97ac-4f62-8f75-855160a20584 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:13.986764 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfc0f4-97ac-4f62-8f75-855160a20584 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:13.987037 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfc0f4-97ac-4f62-8f75-855160a20584 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:13.998569 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebfc0f4-97ac-4f62-8f75-855160a20584 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:14.000029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebfc0f4-97ac-4f62-8f75-855160a20584 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:14.002465 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5ee8c-be42-4c79-a858-c727bc640cf3 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:14.013770 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:14.017884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-9ebfc0f4-97ac-4f62-8f75-855160a20584 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:14.017884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:14.017884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:14.017884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:14.017884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:14.017884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:14.017884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:14.017884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:14.017884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:14.017884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:14.017884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:14.017884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:14.017884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:14.017884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:14.017884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:14.019019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:14.019019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:14.019019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:14.019019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:14.019019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:14.019019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:14.019019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:14.019019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:14.019019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:14.019019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:14.019019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:14.019019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:14.019019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:14.019019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:14.019019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:14.019019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:14.019019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:14.019976 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:14.019976 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:14.019976 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:14.019976 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:14.019976 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 330/670] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:14.020276 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebfc0f4-97ac-4f62-8f75-855160a20584 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:14.044371 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f472dd0-27ba-4105-95cc-ac0d0770398c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:14.045025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f472dd0-27ba-4105-95cc-ac0d0770398c None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:14.045309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f472dd0-27ba-4105-95cc-ac0d0770398c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:14.045583 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f472dd0-27ba-4105-95cc-ac0d0770398c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:14.046403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f472dd0-27ba-4105-95cc-ac0d0770398c None admin] RBAC: Adding query filter params (role.id=a72e5464b7b746978805326c224753bc, scope.project.id=f67bad575f8b4a47bee7c314497cb2fe, user.id=2ae726fbc9cf40e68c7587552a118a7a) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:14.046712 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f472dd0-27ba-4105-95cc-ac0d0770398c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:14.048148 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f472dd0-27ba-4105-95cc-ac0d0770398c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:14.063700 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 341/671] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:08:13 2022] GET /identity/v3/role_assignments?user.id=2ae726fbc9cf40e68c7587552a118a7a&scope.project.id=f67bad575f8b4a47bee7c314497cb2fe&role.id=a72e5464b7b746978805326c224753bc => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:14.065388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f472dd0-27ba-4105-95cc-ac0d0770398c None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:14.110647 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:14.148700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebfc0f4-97ac-4f62-8f75-855160a20584 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:14.198227 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3237512-3b98-4e52-9083-fde059a05981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:14.199489 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3237512-3b98-4e52-9083-fde059a05981 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:14.200003 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3237512-3b98-4e52-9083-fde059a05981 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:14.200483 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3237512-3b98-4e52-9083-fde059a05981 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:14.214155 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3237512-3b98-4e52-9083-fde059a05981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:14.216074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f472dd0-27ba-4105-95cc-ac0d0770398c None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:14.216525 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3237512-3b98-4e52-9083-fde059a05981 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:14.242900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-a3237512-3b98-4e52-9083-fde059a05981 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:14.242900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:14.242900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:14.242900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:14.242900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:14.242900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:14.242900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:14.242900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:14.242900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:14.242900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:14.242900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:14.242900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:14.242900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:14.242900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:14.242900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:14.243948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:14.243948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:14.243948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:14.243948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:14.243948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:14.243948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:14.243948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:14.243948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:14.243948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:14.243948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:14.243948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:14.243948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:14.243948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:14.243948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:14.243948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:14.243948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:14.243948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:14.244850 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:14.244850 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:14.244850 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:14.244850 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:14.247403 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 331/672] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:14.248644 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3237512-3b98-4e52-9083-fde059a05981 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:14.263849 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6caa3e-da33-4f30-9119-792d7116985e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:14.265079 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6caa3e-da33-4f30-9119-792d7116985e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:14.265568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6caa3e-da33-4f30-9119-792d7116985e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:14.266057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6caa3e-da33-4f30-9119-792d7116985e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:14.283079 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6caa3e-da33-4f30-9119-792d7116985e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:14.285358 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6caa3e-da33-4f30-9119-792d7116985e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:14.307319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-fb6caa3e-da33-4f30-9119-792d7116985e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:14.307319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:14.307319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:14.307319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:14.307319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:14.307319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:14.307319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:14.307319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:14.307319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:14.307319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:14.307319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:14.307319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:14.307319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:14.307319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:14.307319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:14.308472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:14.308472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:14.308472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:14.308472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:14.308472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:14.308472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:14.308472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:14.308472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:14.308472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:14.308472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:14.308472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:14.308472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:14.308472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:14.308472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:14.308472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:14.308472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:14.308472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:14.309589 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:14.309589 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:14.309589 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:14.309589 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:14.311609 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 342/673] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:14.312546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6caa3e-da33-4f30-9119-792d7116985e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:14.353470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:14.376137 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3237512-3b98-4e52-9083-fde059a05981 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:14.409660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:14.423231 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf62f1-b295-48ff-9d93-51cf2a874e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:14.424170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf62f1-b295-48ff-9d93-51cf2a874e18 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:14.424645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf62f1-b295-48ff-9d93-51cf2a874e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:14.425097 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf62f1-b295-48ff-9d93-51cf2a874e18 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:14.426014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf62f1-b295-48ff-9d93-51cf2a874e18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:14.426515 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf62f1-b295-48ff-9d93-51cf2a874e18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:14.428147 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf62f1-b295-48ff-9d93-51cf2a874e18 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:14.441684 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 332/674] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:14.441847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6caa3e-da33-4f30-9119-792d7116985e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:14.442847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf62f1-b295-48ff-9d93-51cf2a874e18 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:14.483491 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda13ea-0e07-42ed-a1b0-0c9695524eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:14.484464 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda13ea-0e07-42ed-a1b0-0c9695524eab None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:14.484928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda13ea-0e07-42ed-a1b0-0c9695524eab None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:14.485378 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda13ea-0e07-42ed-a1b0-0c9695524eab None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:14.486274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda13ea-0e07-42ed-a1b0-0c9695524eab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:14.486776 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda13ea-0e07-42ed-a1b0-0c9695524eab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:14.488480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda13ea-0e07-42ed-a1b0-0c9695524eab None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:14.501855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 343/675] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:14.502991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda13ea-0e07-42ed-a1b0-0c9695524eab None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:14.571261 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf62f1-b295-48ff-9d93-51cf2a874e18 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:14.593759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:14.611688 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda13ea-0e07-42ed-a1b0-0c9695524eab None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:14.615328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d881f1-5d81-4680-9b97-d77020b82d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:14.616100 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d881f1-5d81-4680-9b97-d77020b82d62 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:14.616359 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d881f1-5d81-4680-9b97-d77020b82d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:14.616624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d881f1-5d81-4680-9b97-d77020b82d62 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:14.617462 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d881f1-5d81-4680-9b97-d77020b82d62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:14.617768 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d881f1-5d81-4680-9b97-d77020b82d62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:14.619140 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d881f1-5d81-4680-9b97-d77020b82d62 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:14.639885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 333/676] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:14.640782 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d881f1-5d81-4680-9b97-d77020b82d62 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:14.655559 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609b2c5-3fcc-415a-b23c-2d668b5b2f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:14.656666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609b2c5-3fcc-415a-b23c-2d668b5b2f3c None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:14.657225 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609b2c5-3fcc-415a-b23c-2d668b5b2f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:14.657706 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609b2c5-3fcc-415a-b23c-2d668b5b2f3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:14.667378 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609b2c5-3fcc-415a-b23c-2d668b5b2f3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:14.669207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609b2c5-3fcc-415a-b23c-2d668b5b2f3c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:14.695205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-9609b2c5-3fcc-415a-b23c-2d668b5b2f3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:14.695205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:14.695205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:14.695205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:14.695205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:14.695205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:14.695205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:14.695205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:14.695205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:14.695205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:14.695205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:14.695205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:14.695205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:08:14.695205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:08:14.695205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:08:14.696198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:08:14.696198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:14.696198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:14.696198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:14.696198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:14.696198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:14.696198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:14.696198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:14.696198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:14.696198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:14.696198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:14.696198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:14.696198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:14.696198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:14.696198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:14.696198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:14.696198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:14.697106 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:08:14.697106 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:08:14.697106 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:08:14.697106 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:08:14.697106 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:08:14.697106 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:08:14.697106 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:14.697106 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:14.699682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 344/677] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:14.700764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609b2c5-3fcc-415a-b23c-2d668b5b2f3c None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:14.701997 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:14.763187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d881f1-5d81-4680-9b97-d77020b82d62 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:14.799982 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926b189-5563-43bd-8abb-36d4d29485f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:14.800788 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926b189-5563-43bd-8abb-36d4d29485f8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:14.801075 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926b189-5563-43bd-8abb-36d4d29485f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:14.801354 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926b189-5563-43bd-8abb-36d4d29485f8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:14.809393 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926b189-5563-43bd-8abb-36d4d29485f8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:14.810822 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926b189-5563-43bd-8abb-36d4d29485f8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:14.811870 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609b2c5-3fcc-415a-b23c-2d668b5b2f3c None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:14.825116 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-1926b189-5563-43bd-8abb-36d4d29485f8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 14:08:14.825116 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:14.825116 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:14.825116 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:14.825116 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:14.825116 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:14.825116 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:14.825116 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:14.825116 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:14.825116 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:14.825116 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:14.825116 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:14.825116 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:08:14.825116 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:08:14.825116 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:08:14.825116 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:08:14.826129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:14.826129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:14.826129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:08:14.826129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:14.826129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:08:14.826129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:14.826129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:14.826129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:14.826129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:14.826129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:14.826129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:14.826129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:14.826129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:14.826129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:14.826129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:14.826129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:14.826129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:14.826129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:14.827114 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:14.827114 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:14.827114 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:08:14.827114 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:08:14.827114 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:08:14.827114 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:08:14.827114 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:08:14.827114 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:08:14.827114 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 14:08:14.827114 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:14.827114 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 334/678] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:08:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:14.827114 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926b189-5563-43bd-8abb-36d4d29485f8 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:14.853329 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b6b0d-fd4b-492d-9ce6-6451df99635e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:14.854673 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b6b0d-fd4b-492d-9ce6-6451df99635e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:14.855159 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b6b0d-fd4b-492d-9ce6-6451df99635e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:14.855705 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b6b0d-fd4b-492d-9ce6-6451df99635e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:14.863498 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286b6b0d-fd4b-492d-9ce6-6451df99635e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:14.865501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286b6b0d-fd4b-492d-9ce6-6451df99635e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:14.883850 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-286b6b0d-fd4b-492d-9ce6-6451df99635e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:14.883850 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:14.883850 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:14.883850 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:14.883850 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:14.883850 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:14.883850 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:14.883850 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:14.883850 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:14.883850 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:14.883850 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:14.883850 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:14.883850 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:08:14.883850 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:08:14.883850 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:08:14.884809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:08:14.884809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:14.884809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:14.884809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:14.884809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:14.884809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:14.884809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:14.884809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:14.884809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:14.884809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:14.884809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:14.884809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:14.884809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:14.884809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:14.884809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:14.884809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:14.884809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:14.885740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:08:14.885740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:08:14.885740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:08:14.885740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:08:14.885740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:08:14.885740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:08:14.885740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:14.885740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:14.893294 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 345/679] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:14.894133 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b6b0d-fd4b-492d-9ce6-6451df99635e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:14.903580 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:14.943070 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926b189-5563-43bd-8abb-36d4d29485f8 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:14.954997 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:14.983827 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d80924-7f76-4c2c-a6eb-f837cbf9d106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:14.984837 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d80924-7f76-4c2c-a6eb-f837cbf9d106 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:14.985007 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d80924-7f76-4c2c-a6eb-f837cbf9d106 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:14.985403 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d80924-7f76-4c2c-a6eb-f837cbf9d106 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:14.986291 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d80924-7f76-4c2c-a6eb-f837cbf9d106 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:14.986728 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d80924-7f76-4c2c-a6eb-f837cbf9d106 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:14.988246 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d80924-7f76-4c2c-a6eb-f837cbf9d106 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:15.007495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 335/680] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:08:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:15.008658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d80924-7f76-4c2c-a6eb-f837cbf9d106 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:15.028350 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b6b0d-fd4b-492d-9ce6-6451df99635e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:15.082848 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e869a5-0bd0-4ca7-b778-de2b2e5df30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:15.084101 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e869a5-0bd0-4ca7-b778-de2b2e5df30e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:15.084650 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e869a5-0bd0-4ca7-b778-de2b2e5df30e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:15.085135 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e869a5-0bd0-4ca7-b778-de2b2e5df30e None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:15.086249 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e869a5-0bd0-4ca7-b778-de2b2e5df30e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:15.087033 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e869a5-0bd0-4ca7-b778-de2b2e5df30e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:15.089609 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e869a5-0bd0-4ca7-b778-de2b2e5df30e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:15.117005 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 346/681] 10.222.0.81 () {56 vars in 1135 bytes} [Tue Jun 7 14:08:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:15.118487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e869a5-0bd0-4ca7-b778-de2b2e5df30e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:15.127169 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d80924-7f76-4c2c-a6eb-f837cbf9d106 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:15.167172 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233f9ea-165c-4722-8a41-924c73632294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:15.168079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233f9ea-165c-4722-8a41-924c73632294 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:15.168475 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233f9ea-165c-4722-8a41-924c73632294 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:15.168834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233f9ea-165c-4722-8a41-924c73632294 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:15.169730 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1233f9ea-165c-4722-8a41-924c73632294 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:15.170140 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1233f9ea-165c-4722-8a41-924c73632294 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:15.171822 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1233f9ea-165c-4722-8a41-924c73632294 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:15.189788 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 336/682] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:08:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:15.190765 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233f9ea-165c-4722-8a41-924c73632294 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:15.205831 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:15.222056 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:15.261727 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e869a5-0bd0-4ca7-b778-de2b2e5df30e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:15.300191 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061c48a-f712-4de5-9dc9-a8211cf5fe1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:15.301200 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061c48a-f712-4de5-9dc9-a8211cf5fe1b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:15.301352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061c48a-f712-4de5-9dc9-a8211cf5fe1b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:15.301737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061c48a-f712-4de5-9dc9-a8211cf5fe1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:15.302706 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a061c48a-f712-4de5-9dc9-a8211cf5fe1b None admin] RBAC: Adding query filter params (role.id=a72e5464b7b746978805326c224753bc, scope.project.id=f67bad575f8b4a47bee7c314497cb2fe, user.id=a6a08eaecbcd41228571739d8387882a) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:15.303198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a061c48a-f712-4de5-9dc9-a8211cf5fe1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:15.305265 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a061c48a-f712-4de5-9dc9-a8211cf5fe1b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:15.315555 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 347/683] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:08:15 2022] GET /identity/v3/role_assignments?user.id=a6a08eaecbcd41228571739d8387882a&scope.project.id=f67bad575f8b4a47bee7c314497cb2fe&role.id=a72e5464b7b746978805326c224753bc => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:15.316360 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233f9ea-165c-4722-8a41-924c73632294 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:15.316661 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061c48a-f712-4de5-9dc9-a8211cf5fe1b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:15.370803 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd6002-a765-4a9e-8245-fc743c97a747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:15.372244 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd6002-a765-4a9e-8245-fc743c97a747 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:15.372456 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd6002-a765-4a9e-8245-fc743c97a747 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:15.372950 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd6002-a765-4a9e-8245-fc743c97a747 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:15.469785 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061c48a-f712-4de5-9dc9-a8211cf5fe1b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:15.504228 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:15.505506 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd6002-a765-4a9e-8245-fc743c97a747 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:15.509125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd6002-a765-4a9e-8245-fc743c97a747 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:15.515523 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84872c-3ccd-4a44-b7ab-996267fbfb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:15.516449 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84872c-3ccd-4a44-b7ab-996267fbfb6b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:15.516846 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84872c-3ccd-4a44-b7ab-996267fbfb6b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:15.517224 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84872c-3ccd-4a44-b7ab-996267fbfb6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:15.518180 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84872c-3ccd-4a44-b7ab-996267fbfb6b None admin] RBAC: Adding query filter params (role.id=a72e5464b7b746978805326c224753bc, scope.project.id=f67bad575f8b4a47bee7c314497cb2fe, user.id=38b1ff21b8e5445b878740a327e4442f) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:15.518623 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84872c-3ccd-4a44-b7ab-996267fbfb6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:15.520249 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84872c-3ccd-4a44-b7ab-996267fbfb6b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:15.530705 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 348/684] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:08:15 2022] GET /identity/v3/role_assignments?user.id=38b1ff21b8e5445b878740a327e4442f&scope.project.id=f67bad575f8b4a47bee7c314497cb2fe&role.id=a72e5464b7b746978805326c224753bc => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:15.539303 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a362b0-488a-4570-adb2-86ec148bf8ea None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:15.539303 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a362b0-488a-4570-adb2-86ec148bf8ea None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:15.539303 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a362b0-488a-4570-adb2-86ec148bf8ea None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:15.539303 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 349/685] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:15.545393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac3e44-cb65-47bf-8b48-a743d621a952 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:15.545717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac3e44-cb65-47bf-8b48-a743d621a952 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:15.545953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac3e44-cb65-47bf-8b48-a743d621a952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:15.566835 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:15.581307 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-e2ac3e44-cb65-47bf-8b48-a743d621a952 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:15.582507 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e2ac3e44-cb65-47bf-8b48-a743d621a952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:15.662751 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 337/686] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:08:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:08:15.665915 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd6002-a765-4a9e-8245-fc743c97a747 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:15.672920 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 350/687] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:08:15.700390 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac3e44-cb65-47bf-8b48-a743d621a952 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:15.785189 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd6002-a765-4a9e-8245-fc743c97a747 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:15.828327 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:15.845222 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24856978-ad1b-41b3-a7cc-f45529e6b993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:15.846045 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24856978-ad1b-41b3-a7cc-f45529e6b993 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:15.846330 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24856978-ad1b-41b3-a7cc-f45529e6b993 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:15.846754 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24856978-ad1b-41b3-a7cc-f45529e6b993 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:15.848418 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac3e44-cb65-47bf-8b48-a743d621a952 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:15.860029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24856978-ad1b-41b3-a7cc-f45529e6b993 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:15.861499 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24856978-ad1b-41b3-a7cc-f45529e6b993 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:15.877692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:15.892426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-24856978-ad1b-41b3-a7cc-f45529e6b993 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:15.892426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:15.892426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:15.892426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:15.892426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:15.892426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:15.892426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:15.892426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:15.892426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:15.892426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:15.892426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:15.892426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:15.892426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:15.892426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:15.892426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:15.893395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:15.893395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:15.893395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:15.893395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:15.893395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:15.893395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:15.893395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:15.893395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:15.893395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:15.893395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:15.893395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:15.893395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:15.893395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:15.893395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:15.893395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:15.893395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:15.893395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:15.894298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:15.894298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:15.894298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:15.894298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:15.894519 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035606d-a3d2-444d-ba1e-6605c1b59bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:15.894615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 338/688] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:15.894686 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035606d-a3d2-444d-ba1e-6605c1b59bdb None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:15.894686 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035606d-a3d2-444d-ba1e-6605c1b59bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:15.895360 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035606d-a3d2-444d-ba1e-6605c1b59bdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:15.899529 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24856978-ad1b-41b3-a7cc-f45529e6b993 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:15.910436 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c035606d-a3d2-444d-ba1e-6605c1b59bdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:15.912138 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c035606d-a3d2-444d-ba1e-6605c1b59bdb None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:15.935049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-c035606d-a3d2-444d-ba1e-6605c1b59bdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:15.935049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:15.935049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:15.935049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:15.935049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:15.935049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:15.935049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:15.935049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:15.935049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:15.935049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:15.935049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:15.935049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:15.935049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:15.935049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:15.935049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:15.937043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:15.937043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:15.937043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:15.937043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:15.937043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:15.937043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:15.937043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:15.937043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:15.937043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:15.937043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:15.937043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:15.937043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:15.937043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:15.937043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:15.937043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:15.937043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:15.937043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:15.939015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:15.939015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:15.939015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:15.939015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:15.939015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 351/689] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:08:15.942343 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035606d-a3d2-444d-ba1e-6605c1b59bdb None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:16.033580 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24856978-ad1b-41b3-a7cc-f45529e6b993 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:16.056594 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035606d-a3d2-444d-ba1e-6605c1b59bdb None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:16.083366 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a857a-36e2-4f87-8483-ca234828da41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:16.084049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a857a-36e2-4f87-8483-ca234828da41 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:16.084661 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a857a-36e2-4f87-8483-ca234828da41 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:16.085009 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a857a-36e2-4f87-8483-ca234828da41 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:16.086316 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a857a-36e2-4f87-8483-ca234828da41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:16.086748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a857a-36e2-4f87-8483-ca234828da41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:16.088823 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a857a-36e2-4f87-8483-ca234828da41 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:16.093649 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb93667-f924-46c9-ae95-3777290885a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:16.094553 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb93667-f924-46c9-ae95-3777290885a8 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:16.094929 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb93667-f924-46c9-ae95-3777290885a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:16.095328 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb93667-f924-46c9-ae95-3777290885a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:16.096263 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb93667-f924-46c9-ae95-3777290885a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:16.096668 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb93667-f924-46c9-ae95-3777290885a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:16.098297 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb93667-f924-46c9-ae95-3777290885a8 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:16.103140 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 339/690] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:16.112107 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a857a-36e2-4f87-8483-ca234828da41 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:16.124572 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 352/691] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:16.131045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb93667-f924-46c9-ae95-3777290885a8 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:16.194723 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:16.206769 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:16.280047 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb93667-f924-46c9-ae95-3777290885a8 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:16.330426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a857a-36e2-4f87-8483-ca234828da41 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:16.370783 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95868e6-de6c-417a-8eae-c7fe3c460856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:16.371710 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95868e6-de6c-417a-8eae-c7fe3c460856 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:16.372039 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95868e6-de6c-417a-8eae-c7fe3c460856 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:16.372227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5677d3b8-f459-4e7d-ae6b-38e7295b1e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:16.372351 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95868e6-de6c-417a-8eae-c7fe3c460856 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:16.373212 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677d3b8-f459-4e7d-ae6b-38e7295b1e53 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:16.373701 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677d3b8-f459-4e7d-ae6b-38e7295b1e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:16.374164 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677d3b8-f459-4e7d-ae6b-38e7295b1e53 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:16.375078 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5677d3b8-f459-4e7d-ae6b-38e7295b1e53 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:16.376763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5677d3b8-f459-4e7d-ae6b-38e7295b1e53 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:16.380588 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95868e6-de6c-417a-8eae-c7fe3c460856 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:16.382254 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95868e6-de6c-417a-8eae-c7fe3c460856 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:16.398431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-c95868e6-de6c-417a-8eae-c7fe3c460856 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:16.398431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:16.398431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:16.398431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:16.398431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:16.398431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:16.398431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:16.398431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:16.398431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:16.398431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:16.398431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:16.398431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:16.398431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:08:16.398431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:08:16.398431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:08:16.399487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:08:16.399487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:16.399487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:16.399487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:16.399487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:16.399487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:16.399487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:16.399487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:16.399487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:16.399487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:16.399487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:16.399487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:16.399487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:16.399487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:16.399487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:16.399487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:16.399487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:16.400379 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:08:16.400379 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:08:16.400379 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:08:16.400379 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:08:16.400379 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:08:16.400379 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:08:16.400379 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:16.400379 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:16.404007 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 340/692] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:08:16.406511 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95868e6-de6c-417a-8eae-c7fe3c460856 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:16.434714 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 353/693] 10.222.0.81 () {60 vars in 1132 bytes} [Tue Jun 7 14:08:16 2022] POST /identity/v3/users => generated 254 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:08:16.521482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:16.524848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95868e6-de6c-417a-8eae-c7fe3c460856 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:16.564364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616163d0-b66a-450e-8818-0684b04ccb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:16.565038 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616163d0-b66a-450e-8818-0684b04ccb79 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:16.565375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616163d0-b66a-450e-8818-0684b04ccb79 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:16.565659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616163d0-b66a-450e-8818-0684b04ccb79 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:16.566457 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616163d0-b66a-450e-8818-0684b04ccb79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:16.566765 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616163d0-b66a-450e-8818-0684b04ccb79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:16.568298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616163d0-b66a-450e-8818-0684b04ccb79 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:16.587088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 341/694] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:08:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:16.588521 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5677d3b8-f459-4e7d-ae6b-38e7295b1e53 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:16.634845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e972a-277b-412b-b2ec-15c4c7f08b4c None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:16.635177 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e972a-277b-412b-b2ec-15c4c7f08b4c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:16.635432 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e972a-277b-412b-b2ec-15c4c7f08b4c None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:16.637351 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 342/695] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:16.642725 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd6fd3-3498-4b67-a697-d81541e2d19b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:16.643014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd6fd3-3498-4b67-a697-d81541e2d19b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:16.643311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd6fd3-3498-4b67-a697-d81541e2d19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:16.671967 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-53fd6fd3-3498-4b67-a697-d81541e2d19b None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:16.673216 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-53fd6fd3-3498-4b67-a697-d81541e2d19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:16.746625 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:16.759513 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5677d3b8-f459-4e7d-ae6b-38e7295b1e53 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:16.801268 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de92c3-783a-49dd-af52-e6a3e5c6905f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:16.802191 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de92c3-783a-49dd-af52-e6a3e5c6905f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:16.802476 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de92c3-783a-49dd-af52-e6a3e5c6905f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:16.802750 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de92c3-783a-49dd-af52-e6a3e5c6905f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:16.804041 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de92c3-783a-49dd-af52-e6a3e5c6905f None admin] RBAC: Adding query filter params (role.id=a72e5464b7b746978805326c224753bc, scope.project.id=f67bad575f8b4a47bee7c314497cb2fe, user.id=da7c0f99060a44eaab743ba71f159e88) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:16.804283 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de92c3-783a-49dd-af52-e6a3e5c6905f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:16.805610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 343/696] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:08:16.805815 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de92c3-783a-49dd-af52-e6a3e5c6905f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:16.815169 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 354/697] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:08:16 2022] GET /identity/v3/role_assignments?user.id=da7c0f99060a44eaab743ba71f159e88&scope.project.id=f67bad575f8b4a47bee7c314497cb2fe&role.id=a72e5464b7b746978805326c224753bc => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:16.832120 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd6fd3-3498-4b67-a697-d81541e2d19b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:16.941041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:16.958886 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd6fd3-3498-4b67-a697-d81541e2d19b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:16.999077 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb13110-6990-481a-889c-eb3219e34d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:16.999937 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb13110-6990-481a-889c-eb3219e34d62 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:17.000218 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb13110-6990-481a-889c-eb3219e34d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:17.000492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb13110-6990-481a-889c-eb3219e34d62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:17.013837 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb13110-6990-481a-889c-eb3219e34d62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:08:17.021617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-7fb13110-6990-481a-889c-eb3219e34d62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:08:17.021617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:17.021617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:17.021617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:17.021617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:17.021617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:17.021617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:17.021617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:17.021617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:17.021617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:17.021617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:17.021617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:17.021617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:08:17.021617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:08:17.021617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:08:17.021617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:08:17.023331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:08:17.023331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:08:17.023331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:08:17.023331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:08:17.023331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:17.023331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:17.023331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:17.023331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:17.023331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:17.023331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:17.023331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:17.023331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:17.023331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:17.023331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:17.023331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:17.023331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:17.024190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:17.024190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:17.024190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:17.024190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:17.024190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:08:17.024190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:08:17.024190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:08:17.024190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:08:17.024190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:08:17.024190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:08:17.024190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:08:17.024190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:17.024190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 344/698] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:08:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:17.029593 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de92c3-783a-49dd-af52-e6a3e5c6905f None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:17.151657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de92c3-783a-49dd-af52-e6a3e5c6905f None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:17.194020 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10796f33-b2e0-4d87-accd-1e36cd0d0246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:17.195046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10796f33-b2e0-4d87-accd-1e36cd0d0246 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:17.195548 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10796f33-b2e0-4d87-accd-1e36cd0d0246 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:17.196034 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10796f33-b2e0-4d87-accd-1e36cd0d0246 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:17.196987 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10796f33-b2e0-4d87-accd-1e36cd0d0246 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:17.197486 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10796f33-b2e0-4d87-accd-1e36cd0d0246 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:17.199043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10796f33-b2e0-4d87-accd-1e36cd0d0246 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:17.208624 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 355/699] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:08:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:17.214378 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb13110-6990-481a-889c-eb3219e34d62 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:17.328030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb13110-6990-481a-889c-eb3219e34d62 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:17.366092 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc28da2-3f2c-45e5-a89c-64f17502637c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:17.367288 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc28da2-3f2c-45e5-a89c-64f17502637c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:17.367770 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc28da2-3f2c-45e5-a89c-64f17502637c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:17.368238 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc28da2-3f2c-45e5-a89c-64f17502637c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:17.380325 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc28da2-3f2c-45e5-a89c-64f17502637c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:17.382602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc28da2-3f2c-45e5-a89c-64f17502637c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:17.402339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-adc28da2-3f2c-45e5-a89c-64f17502637c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:17.402339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:17.402339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:17.402339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:17.402339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:17.402339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:17.402339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:17.402339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:17.402339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:17.402339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:17.402339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:17.402339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:17.402339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:17.402339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:17.402339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:17.403350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:17.403350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:17.403350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:17.403350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:17.403350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:17.403350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:17.403350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:17.403350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:17.403350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:17.403350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:17.403350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:17.403350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:17.403350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:17.403350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:17.403350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:17.403350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:17.403350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:17.404252 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:17.404252 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:17.404252 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:17.404252 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:17.406550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 345/700] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:08:17.412098 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10796f33-b2e0-4d87-accd-1e36cd0d0246 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:17.433169 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:17.527663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10796f33-b2e0-4d87-accd-1e36cd0d0246 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:17.576241 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d8206-c56a-4364-abe6-f8092900b8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:17.577307 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d8206-c56a-4364-abe6-f8092900b8ac None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:17.577766 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d8206-c56a-4364-abe6-f8092900b8ac None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:17.578218 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d8206-c56a-4364-abe6-f8092900b8ac None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:17.579486 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d8206-c56a-4364-abe6-f8092900b8ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:17.579998 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d8206-c56a-4364-abe6-f8092900b8ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:17.582351 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d8206-c56a-4364-abe6-f8092900b8ac None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:17.602920 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 356/701] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:17.608823 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc28da2-3f2c-45e5-a89c-64f17502637c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:17.612034 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:17.739544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc28da2-3f2c-45e5-a89c-64f17502637c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:17.794889 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa185a3-eb15-47e7-aac8-606b04426d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:17.795627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa185a3-eb15-47e7-aac8-606b04426d65 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:17.795910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa185a3-eb15-47e7-aac8-606b04426d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:17.796189 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa185a3-eb15-47e7-aac8-606b04426d65 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:17.808438 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa185a3-eb15-47e7-aac8-606b04426d65 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:17.810329 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa185a3-eb15-47e7-aac8-606b04426d65 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:17.833136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-caa185a3-eb15-47e7-aac8-606b04426d65 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:08:17.833136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:17.833136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:17.833136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:17.833136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:17.833136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:17.833136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:17.833136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:17.833136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:17.833136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:17.833136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:17.833136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:17.833136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:08:17.833136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:08:17.833136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:08:17.833136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:08:17.834139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:17.834139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:17.834139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:08:17.834139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:17.834139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:08:17.834139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:17.834139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:17.834139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:17.834139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:17.834139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:17.834139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:17.834139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:17.834139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:17.834139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:17.834139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:17.834139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:17.834139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:17.834139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:17.835695 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:17.835695 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:17.835695 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:08:17.835695 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:08:17.835695 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:08:17.835695 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:08:17.835695 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:08:17.835695 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:08:17.835695 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:08:17.835695 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:17.838484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 346/702] 10.222.0.81 () {56 vars in 1093 bytes} [Tue Jun 7 14:08:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:17.843661 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d8206-c56a-4364-abe6-f8092900b8ac None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:17.965172 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:17.993335 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d8206-c56a-4364-abe6-f8092900b8ac None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:18.039589 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a086096-8522-43a1-a675-b9c5cd243d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:18.040260 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a086096-8522-43a1-a675-b9c5cd243d1e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:18.040538 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a086096-8522-43a1-a675-b9c5cd243d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:18.040809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a086096-8522-43a1-a675-b9c5cd243d1e None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:18.041642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a086096-8522-43a1-a675-b9c5cd243d1e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:18.041945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a086096-8522-43a1-a675-b9c5cd243d1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:18.043397 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a086096-8522-43a1-a675-b9c5cd243d1e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:18.061395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 357/703] 10.222.0.81 () {56 vars in 1133 bytes} [Tue Jun 7 14:08:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:18.068665 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa185a3-eb15-47e7-aac8-606b04426d65 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:18.182129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:18.243058 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa185a3-eb15-47e7-aac8-606b04426d65 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:18.285435 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5089d-603a-456e-830d-7d8920843033 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:18.285751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5089d-603a-456e-830d-7d8920843033 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:18.285989 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5089d-603a-456e-830d-7d8920843033 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:18.288241 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 358/704] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:18.289676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6cb7b-eb23-4119-b69f-cac92d3a5cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:18.290343 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6cb7b-eb23-4119-b69f-cac92d3a5cfb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:18.290629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6cb7b-eb23-4119-b69f-cac92d3a5cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:18.290906 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6cb7b-eb23-4119-b69f-cac92d3a5cfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:18.293383 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d4115-b2f0-4eb1-91e1-408e6b36ed68 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:18.293669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d4115-b2f0-4eb1-91e1-408e6b36ed68 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:18.293953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d4115-b2f0-4eb1-91e1-408e6b36ed68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:18.311644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-364d4115-b2f0-4eb1-91e1-408e6b36ed68 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:18.312743 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-364d4115-b2f0-4eb1-91e1-408e6b36ed68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:18.421384 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6cb7b-eb23-4119-b69f-cac92d3a5cfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:18.422897 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6cb7b-eb23-4119-b69f-cac92d3a5cfb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:18.433454 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 359/705] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:08:18.460859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d4115-b2f0-4eb1-91e1-408e6b36ed68 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:18.515543 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:18.547143 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:18.566914 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 347/706] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:08:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:08:18.568498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd98ed-dad9-4d91-828c-377783489e12 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:18.568822 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd98ed-dad9-4d91-828c-377783489e12 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:18.569104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd98ed-dad9-4d91-828c-377783489e12 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:18.575266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 348/707] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:18.575266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd98ed-dad9-4d91-828c-377783489e12 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:18.596122 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d4115-b2f0-4eb1-91e1-408e6b36ed68 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:18.634547 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d5f6c3-cf05-4bf0-9b1d-97b4d91bf837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:18.635754 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5f6c3-cf05-4bf0-9b1d-97b4d91bf837 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:18.636215 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5f6c3-cf05-4bf0-9b1d-97b4d91bf837 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:18.636659 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5f6c3-cf05-4bf0-9b1d-97b4d91bf837 None admin] PATH_INFO: `/v3/services/network` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:18.637480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d5f6c3-cf05-4bf0-9b1d-97b4d91bf837 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:18.639123 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d5f6c3-cf05-4bf0-9b1d-97b4d91bf837 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:18.654525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-b8d5f6c3-cf05-4bf0-9b1d-97b4d91bf837 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 14:08:18.654525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:18.654525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:08:18.654525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:08:18.654525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:08:18.654525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:08:18.654525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:08:18.654525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:18.654525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 14:08:18.654525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:18.654525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:08:18.654525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:18.654525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:18.654525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:18.654525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:18.654525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:18.654525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:18.655702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:18.655702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:18.655702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:18.655702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:18.655702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:18.655702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:18.655702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:08:18.655702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:08:18.655702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:08:18.655702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:08:18.655702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:18.655702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:18.655702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:18.655702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:18.655702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:18.655702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:18.656599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:18.656599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:18.656599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:18.656599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:18.656599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:18.656599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:18.656599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:18.656599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:18.656599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:18.656599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:18.656599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:08:18.656599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:18.656599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 14:08:18.656599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:18.659290 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 360/708] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:18 2022] GET /identity/v3/services/network => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:08:18.660776 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f600ff3-b08c-4e34-88b0-e558cf98692d None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:18.661436 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f600ff3-b08c-4e34-88b0-e558cf98692d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:18.661720 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f600ff3-b08c-4e34-88b0-e558cf98692d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:18.679760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-7f600ff3-b08c-4e34-88b0-e558cf98692d None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:18.680947 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7f600ff3-b08c-4e34-88b0-e558cf98692d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:18.696925 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd98ed-dad9-4d91-828c-377783489e12 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:18.742414 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b23df-913b-4985-af26-1dd4b90727b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:18.743810 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b23df-913b-4985-af26-1dd4b90727b1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:18.744316 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b23df-913b-4985-af26-1dd4b90727b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:18.744789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b23df-913b-4985-af26-1dd4b90727b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:18.755277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 361/709] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:08:18.755277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f600ff3-b08c-4e34-88b0-e558cf98692d None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:18.769126 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b23df-913b-4985-af26-1dd4b90727b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:18.770998 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b23df-913b-4985-af26-1dd4b90727b1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:18.802041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-a73b23df-913b-4985-af26-1dd4b90727b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:18.802041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:18.802041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:18.802041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:18.802041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:18.802041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:18.802041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:18.802041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:18.802041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:18.802041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:18.802041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:18.802041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:18.802041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:18.802041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:18.802041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:18.803550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:18.803550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:18.803550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:18.803550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:18.803550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:18.803550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:18.803550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:18.803550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:18.803550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:18.803550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:18.803550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:18.803550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:18.803550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:18.803550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:18.803550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:18.803550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:18.803550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:18.804659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:18.804659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:18.804659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:18.804659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:18.807051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 349/710] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:18.808173 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b23df-913b-4985-af26-1dd4b90727b1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:18.812540 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:18.827145 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:18.856134 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f600ff3-b08c-4e34-88b0-e558cf98692d None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:18.894233 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f351b-b1cb-4a49-b34a-5ed624324150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:18.894907 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f351b-b1cb-4a49-b34a-5ed624324150 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:18.895200 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f351b-b1cb-4a49-b34a-5ed624324150 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:18.895505 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f351b-b1cb-4a49-b34a-5ed624324150 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:18.896276 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f351b-b1cb-4a49-b34a-5ed624324150 None admin] RBAC: Adding query filter params (name=network) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:18.896589 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f351b-b1cb-4a49-b34a-5ed624324150 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:18.897995 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f351b-b1cb-4a49-b34a-5ed624324150 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:18.907356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 362/711] 10.222.0.81 () {56 vars in 1112 bytes} [Tue Jun 7 14:08:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:18.907356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f351b-b1cb-4a49-b34a-5ed624324150 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:18.940955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b23df-913b-4985-af26-1dd4b90727b1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:18.983386 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9bbe53-53cc-4ff8-8c16-5a95f96bcbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:18.984554 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9bbe53-53cc-4ff8-8c16-5a95f96bcbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:18.985037 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9bbe53-53cc-4ff8-8c16-5a95f96bcbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:18.985511 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9bbe53-53cc-4ff8-8c16-5a95f96bcbe1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:18.986381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9bbe53-53cc-4ff8-8c16-5a95f96bcbe1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:18.987985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9bbe53-53cc-4ff8-8c16-5a95f96bcbe1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:19.003106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-fa9bbe53-53cc-4ff8-8c16-5a95f96bcbe1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 14:08:19.003106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:19.003106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:08:19.003106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:08:19.003106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:08:19.003106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:08:19.003106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:08:19.003106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:19.003106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 14:08:19.003106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:19.003106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:08:19.003106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:19.003106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:19.003106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:19.003106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:19.003106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:19.004235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:19.004235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:19.004235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:19.004235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:19.004235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:19.004235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:19.004235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:19.004235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:08:19.004235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:08:19.004235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:08:19.004235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:08:19.004235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:19.004235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:19.004235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:19.004235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:19.004235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:19.005079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:19.005079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:19.005079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:19.005079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:19.005079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:19.005079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:19.005079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:19.005079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:19.005079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:19.005079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:19.005079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:19.005079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:08:19.005079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:19.005079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 14:08:19.005079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:19.011458 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 350/712] 10.222.0.81 () {56 vars in 1129 bytes} [Tue Jun 7 14:08:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:19.012526 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9bbe53-53cc-4ff8-8c16-5a95f96bcbe1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:19.028844 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f351b-b1cb-4a49-b34a-5ed624324150 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:19.062944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:19.079461 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58e3bd-6770-46bc-9e20-4642770fd858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:19.080069 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58e3bd-6770-46bc-9e20-4642770fd858 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:19.080360 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58e3bd-6770-46bc-9e20-4642770fd858 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:19.080642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58e3bd-6770-46bc-9e20-4642770fd858 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:19.081379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd58e3bd-6770-46bc-9e20-4642770fd858 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:19.081690 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd58e3bd-6770-46bc-9e20-4642770fd858 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:19.083055 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd58e3bd-6770-46bc-9e20-4642770fd858 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:19.100901 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 363/713] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:19.101764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58e3bd-6770-46bc-9e20-4642770fd858 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:19.113739 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:19.140727 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9bbe53-53cc-4ff8-8c16-5a95f96bcbe1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:19.185616 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8313e36-0a0b-412c-a456-165b183bd6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:19.186605 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8313e36-0a0b-412c-a456-165b183bd6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:19.187181 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8313e36-0a0b-412c-a456-165b183bd6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:19.187679 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8313e36-0a0b-412c-a456-165b183bd6c7 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:19.188700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8313e36-0a0b-412c-a456-165b183bd6c7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:19.189217 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8313e36-0a0b-412c-a456-165b183bd6c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:19.190792 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8313e36-0a0b-412c-a456-165b183bd6c7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:19.202537 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 351/714] 10.222.0.81 () {56 vars in 1112 bytes} [Tue Jun 7 14:08:19 2022] GET /identity/v3/services?type=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:19.203426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8313e36-0a0b-412c-a456-165b183bd6c7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:19.237885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58e3bd-6770-46bc-9e20-4642770fd858 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:19.285170 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc58f67-82fa-4a0a-911a-9e17000cc66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:19.285849 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc58f67-82fa-4a0a-911a-9e17000cc66c None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:19.286191 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc58f67-82fa-4a0a-911a-9e17000cc66c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:19.286465 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc58f67-82fa-4a0a-911a-9e17000cc66c None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:19.287241 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc58f67-82fa-4a0a-911a-9e17000cc66c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:19.288141 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc58f67-82fa-4a0a-911a-9e17000cc66c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:19.289588 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc58f67-82fa-4a0a-911a-9e17000cc66c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:19.295125 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 364/715] 10.222.0.81 () {56 vars in 1124 bytes} [Tue Jun 7 14:08:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:19.300733 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc58f67-82fa-4a0a-911a-9e17000cc66c None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:19.332427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8313e36-0a0b-412c-a456-165b183bd6c7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:19.378033 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e1e17-2040-411e-8283-c983faf1983c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:19.379518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e1e17-2040-411e-8283-c983faf1983c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:19.380029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e1e17-2040-411e-8283-c983faf1983c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:19.380509 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e1e17-2040-411e-8283-c983faf1983c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:19.390244 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e1e17-2040-411e-8283-c983faf1983c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:19.392770 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e1e17-2040-411e-8283-c983faf1983c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:19.410777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-7d6e1e17-2040-411e-8283-c983faf1983c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:19.410777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:19.410777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:19.410777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:19.410777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:19.410777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:19.410777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:19.410777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:19.410777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:19.410777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:19.410777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:19.410777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:19.410777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:08:19.410777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:08:19.410777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:08:19.411780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:08:19.411780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:19.411780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:19.411780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:19.411780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:19.411780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:19.411780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:19.411780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:19.411780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:19.411780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:19.411780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:19.411780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:19.411780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:19.411780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:19.411780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:19.411780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:19.411780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:19.412891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:08:19.412891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:08:19.412891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:08:19.412891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:08:19.412891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:08:19.412891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:08:19.412891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:19.412891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:19.415565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 352/716] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:19.417163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:19.421705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e1e17-2040-411e-8283-c983faf1983c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:19.428868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:19.512139 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc58f67-82fa-4a0a-911a-9e17000cc66c None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:19.550011 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64fa32-f93a-4699-b91b-4f5889a65187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:19.551269 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64fa32-f93a-4699-b91b-4f5889a65187 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:19.552159 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64fa32-f93a-4699-b91b-4f5889a65187 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:19.552855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64fa32-f93a-4699-b91b-4f5889a65187 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:19.555418 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c64fa32-f93a-4699-b91b-4f5889a65187 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:19.556097 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c64fa32-f93a-4699-b91b-4f5889a65187 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:19.558355 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c64fa32-f93a-4699-b91b-4f5889a65187 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:19.568093 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 365/717] 10.222.0.81 () {56 vars in 1124 bytes} [Tue Jun 7 14:08:19 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:19.603090 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e1e17-2040-411e-8283-c983faf1983c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:19.642662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58512f80-1fea-4c19-ba1d-a8dfa64ed617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:19.643378 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58512f80-1fea-4c19-ba1d-a8dfa64ed617 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:19.643663 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58512f80-1fea-4c19-ba1d-a8dfa64ed617 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:19.643937 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58512f80-1fea-4c19-ba1d-a8dfa64ed617 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:19.644737 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58512f80-1fea-4c19-ba1d-a8dfa64ed617 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:19.645043 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58512f80-1fea-4c19-ba1d-a8dfa64ed617 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:19.646484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58512f80-1fea-4c19-ba1d-a8dfa64ed617 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:19.659847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 353/718] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:08:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:19.666369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64fa32-f93a-4699-b91b-4f5889a65187 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:19.680083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189a94b-0bfe-4e82-a99e-c753040ea9d8 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:19.680477 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189a94b-0bfe-4e82-a99e-c753040ea9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:19.680852 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189a94b-0bfe-4e82-a99e-c753040ea9d8 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:19.687170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 354/719] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:19.690370 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d379764-0d17-4d5a-9f60-656ac8d272f2 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:19.692775 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d379764-0d17-4d5a-9f60-656ac8d272f2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:19.693285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d379764-0d17-4d5a-9f60-656ac8d272f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:19.730242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1d379764-0d17-4d5a-9f60-656ac8d272f2 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:19.732270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1d379764-0d17-4d5a-9f60-656ac8d272f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:19.811128 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64fa32-f93a-4699-b91b-4f5889a65187 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:19.827748 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:19.854564 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca62049-3caf-4680-a4de-6c0e5aabd1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:19.855168 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca62049-3caf-4680-a4de-6c0e5aabd1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:19.855664 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca62049-3caf-4680-a4de-6c0e5aabd1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:19.855960 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca62049-3caf-4680-a4de-6c0e5aabd1a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:19.856876 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca62049-3caf-4680-a4de-6c0e5aabd1a6 None admin] RBAC: Adding query filter params (role.id=82bbe3651b9b4f378fd150d93357b2b1, scope.project.id=f67bad575f8b4a47bee7c314497cb2fe, user.id=2ae726fbc9cf40e68c7587552a118a7a) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:19.857201 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca62049-3caf-4680-a4de-6c0e5aabd1a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:19.858665 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca62049-3caf-4680-a4de-6c0e5aabd1a6 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:19.867815 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 366/720] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:08:19 2022] GET /identity/v3/role_assignments?user.id=2ae726fbc9cf40e68c7587552a118a7a&scope.project.id=f67bad575f8b4a47bee7c314497cb2fe&role.id=82bbe3651b9b4f378fd150d93357b2b1 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:19.917199 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:19.938049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 355/721] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:08:19.984871 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca62049-3caf-4680-a4de-6c0e5aabd1a6 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:20.088783 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f75bf-519f-4a4a-bf0f-f2daf30eaf3a None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:20.089445 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f75bf-519f-4a4a-bf0f-f2daf30eaf3a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:20.089924 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f75bf-519f-4a4a-bf0f-f2daf30eaf3a None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:20.093214 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 356/722] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:20.098644 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b59d1f-ca79-4447-bd8f-40fe097b5f33 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:20.099040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b59d1f-ca79-4447-bd8f-40fe097b5f33 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:20.099456 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b59d1f-ca79-4447-bd8f-40fe097b5f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:20.122075 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca62049-3caf-4680-a4de-6c0e5aabd1a6 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:20.126456 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d6b59d1f-ca79-4447-bd8f-40fe097b5f33 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:20.132307 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d6b59d1f-ca79-4447-bd8f-40fe097b5f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:20.178979 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a042858-ff13-405f-9841-9e2359903568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:20.179808 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a042858-ff13-405f-9841-9e2359903568 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:20.180089 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a042858-ff13-405f-9841-9e2359903568 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:20.180358 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a042858-ff13-405f-9841-9e2359903568 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:20.192015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a042858-ff13-405f-9841-9e2359903568 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:08:20.200835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-8a042858-ff13-405f-9841-9e2359903568 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:20.200835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:20.200835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:20.200835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:20.200835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:20.200835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:20.200835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:20.200835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:20.200835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:20.200835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:20.200835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:20.200835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:20.200835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:08:20.200835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:08:20.200835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:08:20.200835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:08:20.201795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:08:20.201795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:08:20.201795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:08:20.201795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:08:20.201795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:20.201795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:20.201795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:20.201795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:20.201795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:20.201795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:20.201795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:20.201795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:20.201795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:20.201795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:20.201795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:20.201795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:20.203102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:20.203102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:20.203102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:20.203102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:20.203102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:08:20.203102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:08:20.203102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:08:20.203102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:08:20.203102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:08:20.203102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:08:20.203102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:20.203102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:20.203102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 367/723] 10.222.0.81 () {56 vars in 1102 bytes} [Tue Jun 7 14:08:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:20.209852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a042858-ff13-405f-9841-9e2359903568 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:20.247579 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:20.282738 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 357/724] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:08:20.307021 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b59d1f-ca79-4447-bd8f-40fe097b5f33 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:20.376063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a042858-ff13-405f-9841-9e2359903568 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:20.441917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce82d1a-2242-4ca4-825b-5cc411ba14c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:20.442950 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce82d1a-2242-4ca4-825b-5cc411ba14c5 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:20.443627 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce82d1a-2242-4ca4-825b-5cc411ba14c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:20.444238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce82d1a-2242-4ca4-825b-5cc411ba14c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:20.445320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce82d1a-2242-4ca4-825b-5cc411ba14c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:20.445939 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce82d1a-2242-4ca4-825b-5cc411ba14c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:20.447658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce82d1a-2242-4ca4-825b-5cc411ba14c5 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:20.461417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 368/725] 10.222.0.81 () {56 vars in 1103 bytes} [Tue Jun 7 14:08:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:20.463073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce82d1a-2242-4ca4-825b-5cc411ba14c5 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:20.483007 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b59d1f-ca79-4447-bd8f-40fe097b5f33 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:20.531804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:20.556271 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fdf63-5f3b-4226-a9f9-25240412e129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:20.557676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fdf63-5f3b-4226-a9f9-25240412e129 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:20.559242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fdf63-5f3b-4226-a9f9-25240412e129 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:20.559912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fdf63-5f3b-4226-a9f9-25240412e129 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:20.580612 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fdf63-5f3b-4226-a9f9-25240412e129 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:08:20.597130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-d80fdf63-5f3b-4226-a9f9-25240412e129 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:08:20.597130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:20.597130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:20.597130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:20.597130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:20.597130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:20.597130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:20.597130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:20.597130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:20.597130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:20.597130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:20.597130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:20.597130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:08:20.597130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:08:20.597130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:08:20.597130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:08:20.598185 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:08:20.598185 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:08:20.598185 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:08:20.598185 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:08:20.598185 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:20.598185 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:20.598185 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:20.598185 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:20.598185 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:20.598185 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:20.598185 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:20.598185 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:20.598185 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:20.598185 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:20.598185 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:20.598185 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:20.599029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:20.599029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:20.599029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:20.599029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:20.599029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:08:20.599029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:08:20.599029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:08:20.599029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:08:20.599029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:08:20.599029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:08:20.599029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:08:20.599029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:20.605017 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 358/726] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:08:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:08:20.609856 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fdf63-5f3b-4226-a9f9-25240412e129 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:20.639098 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:20.647498 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce82d1a-2242-4ca4-825b-5cc411ba14c5 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:20.700144 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad9e88-8610-42e0-ab9b-cd3bf076563a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:20.701317 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad9e88-8610-42e0-ab9b-cd3bf076563a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:20.701816 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad9e88-8610-42e0-ab9b-cd3bf076563a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:20.702297 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad9e88-8610-42e0-ab9b-cd3bf076563a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:20.715872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dad9e88-8610-42e0-ab9b-cd3bf076563a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:20.717913 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dad9e88-8610-42e0-ab9b-cd3bf076563a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:20.753692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-1dad9e88-8610-42e0-ab9b-cd3bf076563a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:20.753692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:20.753692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:20.753692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:20.753692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:20.753692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:20.753692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:20.753692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:20.753692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:20.753692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:20.753692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:20.753692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:20.753692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:20.753692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:20.753692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:20.754669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:20.754669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:20.754669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:20.754669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:20.754669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:20.754669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:20.754669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:20.754669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:20.754669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:20.754669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:20.754669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:20.754669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:20.754669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:20.754669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:20.754669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:20.754669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:20.754669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:20.755602 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:20.755602 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:20.755602 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:20.755602 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:20.757828 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 369/727] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:08:20.763634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad9e88-8610-42e0-ab9b-cd3bf076563a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:20.817286 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fdf63-5f3b-4226-a9f9-25240412e129 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:20.862967 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dd124-25ff-47a1-8173-59570d7841a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:20.864020 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dd124-25ff-47a1-8173-59570d7841a1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:20.864502 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dd124-25ff-47a1-8173-59570d7841a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:20.864973 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dd124-25ff-47a1-8173-59570d7841a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:20.866719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470dd124-25ff-47a1-8173-59570d7841a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:20.867269 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470dd124-25ff-47a1-8173-59570d7841a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:20.869493 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470dd124-25ff-47a1-8173-59570d7841a1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:20.890277 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 359/728] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:08:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:20.895631 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dd124-25ff-47a1-8173-59570d7841a1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:20.991578 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:21.010537 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad9e88-8610-42e0-ab9b-cd3bf076563a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:21.074135 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc61ff-5db0-4f0f-bbce-410084c3e086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:21.075163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc61ff-5db0-4f0f-bbce-410084c3e086 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:21.075680 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc61ff-5db0-4f0f-bbce-410084c3e086 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:21.076296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc61ff-5db0-4f0f-bbce-410084c3e086 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:21.077311 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc61ff-5db0-4f0f-bbce-410084c3e086 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:21.077807 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc61ff-5db0-4f0f-bbce-410084c3e086 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:21.079389 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc61ff-5db0-4f0f-bbce-410084c3e086 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:21.079565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dd124-25ff-47a1-8173-59570d7841a1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:21.101978 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 370/729] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:21.108242 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc61ff-5db0-4f0f-bbce-410084c3e086 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:21.132129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6eb362-7b1c-41de-b1e1-720dbd991b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:21.133605 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6eb362-7b1c-41de-b1e1-720dbd991b41 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:21.133886 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6eb362-7b1c-41de-b1e1-720dbd991b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:21.134159 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6eb362-7b1c-41de-b1e1-720dbd991b41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:21.134998 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:21.159395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6eb362-7b1c-41de-b1e1-720dbd991b41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:21.160787 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6eb362-7b1c-41de-b1e1-720dbd991b41 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:21.194753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-ff6eb362-7b1c-41de-b1e1-720dbd991b41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:21.194753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:21.194753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:21.194753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:21.194753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:21.194753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:21.194753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:21.194753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:21.194753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:21.194753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:21.194753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:21.194753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:21.194753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:21.194753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:21.194753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:21.195748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:21.195748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:21.195748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:21.195748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:21.195748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:21.195748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:21.195748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:21.195748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:21.195748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:21.195748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:21.195748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:21.195748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:21.195748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:21.195748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:21.195748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:21.195748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:21.195748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:21.196656 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:21.196656 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:21.196656 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:21.196656 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:21.196656 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 360/730] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:21.202243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6eb362-7b1c-41de-b1e1-720dbd991b41 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:21.248996 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc61ff-5db0-4f0f-bbce-410084c3e086 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:21.296415 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68a556-a6e7-4dbc-bb6d-36c1eedbd198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:21.297235 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68a556-a6e7-4dbc-bb6d-36c1eedbd198 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:21.297521 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68a556-a6e7-4dbc-bb6d-36c1eedbd198 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:21.297795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68a556-a6e7-4dbc-bb6d-36c1eedbd198 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:21.315259 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a68a556-a6e7-4dbc-bb6d-36c1eedbd198 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:21.316717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a68a556-a6e7-4dbc-bb6d-36c1eedbd198 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:21.343957 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-0a68a556-a6e7-4dbc-bb6d-36c1eedbd198 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 14:08:21.343957 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:21.343957 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:21.343957 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:21.343957 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:21.343957 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:21.343957 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:21.343957 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:21.343957 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:21.343957 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:21.343957 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:21.343957 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:21.343957 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:08:21.343957 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:08:21.343957 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:08:21.343957 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:08:21.346131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:21.346131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:21.346131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:08:21.346131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:21.346131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:08:21.346131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:21.346131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:21.346131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:21.346131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:21.346131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:21.346131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:21.346131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:21.346131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:21.346131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:21.346131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:21.346131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:21.346131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:21.346131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:21.348319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:21.348319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:21.348319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:08:21.348319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:08:21.348319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:08:21.348319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:08:21.348319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:08:21.348319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:08:21.348319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 14:08:21.348319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:21.352325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 371/731] 10.222.0.81 () {56 vars in 1117 bytes} [Tue Jun 7 14:08:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:08:21.360209 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68a556-a6e7-4dbc-bb6d-36c1eedbd198 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:21.372004 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6eb362-7b1c-41de-b1e1-720dbd991b41 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:21.375597 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:21.414836 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641367eb-7d71-47c8-884f-d9985684da4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:21.415447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641367eb-7d71-47c8-884f-d9985684da4d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:21.415733 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641367eb-7d71-47c8-884f-d9985684da4d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:21.416014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641367eb-7d71-47c8-884f-d9985684da4d None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:21.416885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641367eb-7d71-47c8-884f-d9985684da4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:21.417195 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641367eb-7d71-47c8-884f-d9985684da4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:21.418566 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641367eb-7d71-47c8-884f-d9985684da4d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:21.432627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 361/732] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:21.437595 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641367eb-7d71-47c8-884f-d9985684da4d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:21.503254 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:21.550849 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68a556-a6e7-4dbc-bb6d-36c1eedbd198 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:21.594185 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f8c9a-1feb-45a2-9aeb-b8b585824b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:21.595052 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f8c9a-1feb-45a2-9aeb-b8b585824b46 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:21.595366 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f8c9a-1feb-45a2-9aeb-b8b585824b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:21.595646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f8c9a-1feb-45a2-9aeb-b8b585824b46 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:21.596655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f8c9a-1feb-45a2-9aeb-b8b585824b46 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:21.596975 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f8c9a-1feb-45a2-9aeb-b8b585824b46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:21.598881 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f8c9a-1feb-45a2-9aeb-b8b585824b46 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:21.635037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 372/733] 10.222.0.81 () {56 vars in 1149 bytes} [Tue Jun 7 14:08:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:21.644611 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f8c9a-1feb-45a2-9aeb-b8b585824b46 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:21.753121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641367eb-7d71-47c8-884f-d9985684da4d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:21.833897 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b553c-c440-41fa-8aaf-663fb4d27d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:21.836476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b553c-c440-41fa-8aaf-663fb4d27d62 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:21.837117 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b553c-c440-41fa-8aaf-663fb4d27d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:21.838674 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b553c-c440-41fa-8aaf-663fb4d27d62 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:21.862078 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b553c-c440-41fa-8aaf-663fb4d27d62 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:21.863987 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b553c-c440-41fa-8aaf-663fb4d27d62 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:21.889760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f8c9a-1feb-45a2-9aeb-b8b585824b46 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:21.905125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-ae0b553c-c440-41fa-8aaf-663fb4d27d62 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 14:08:21.905125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:21.905125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:21.905125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:21.905125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:21.905125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:21.905125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:21.905125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:21.905125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:21.905125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:21.905125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:21.905125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:21.905125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:08:21.905125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:08:21.905125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:08:21.905125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:08:21.906249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:21.906249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:21.906249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:08:21.906249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:21.906249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:08:21.906249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:21.906249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:21.906249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:21.906249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:21.906249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:21.906249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:21.906249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:21.906249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:21.906249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:21.906249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:21.906249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:21.906249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:21.906249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:21.907205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:21.907205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:21.907205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:08:21.907205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:08:21.907205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:08:21.907205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:08:21.907205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:08:21.907205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:08:21.907205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 14:08:21.907205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:21.914814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 362/734] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:08:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 477 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:21.920542 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b553c-c440-41fa-8aaf-663fb4d27d62 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:21.959824 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:21.964780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:21.986918 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99165f40-6c16-4707-b7ef-a0561c438053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:21.990043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99165f40-6c16-4707-b7ef-a0561c438053 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:21.991606 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99165f40-6c16-4707-b7ef-a0561c438053 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:21.992363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99165f40-6c16-4707-b7ef-a0561c438053 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:22.113670 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99165f40-6c16-4707-b7ef-a0561c438053 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:22.115055 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99165f40-6c16-4707-b7ef-a0561c438053 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:22.127293 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b553c-c440-41fa-8aaf-663fb4d27d62 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:22.183256 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05737eed-9600-4f19-a30c-6359a74e7595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:22.184296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05737eed-9600-4f19-a30c-6359a74e7595 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:22.184783 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05737eed-9600-4f19-a30c-6359a74e7595 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:22.185280 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05737eed-9600-4f19-a30c-6359a74e7595 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:22.186333 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05737eed-9600-4f19-a30c-6359a74e7595 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:22.186843 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05737eed-9600-4f19-a30c-6359a74e7595 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:22.188543 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05737eed-9600-4f19-a30c-6359a74e7595 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:22.211835 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 363/735] 10.222.0.81 () {56 vars in 1135 bytes} [Tue Jun 7 14:08:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:22.213433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d026d0-8d88-49a3-aa28-58039a75b0da None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:22.213883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d026d0-8d88-49a3-aa28-58039a75b0da None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:22.214304 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d026d0-8d88-49a3-aa28-58039a75b0da None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:22.216002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 364/736] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:22.216879 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d026d0-8d88-49a3-aa28-58039a75b0da None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:22.251267 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 373/737] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:08:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:08:22.251267 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bb5a7-6725-409d-9fec-e231056f3bbb None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:22.251267 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bb5a7-6725-409d-9fec-e231056f3bbb None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:22.251267 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bb5a7-6725-409d-9fec-e231056f3bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:22.272585 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-364bb5a7-6725-409d-9fec-e231056f3bbb None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:22.273822 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-364bb5a7-6725-409d-9fec-e231056f3bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:22.300227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:22.327261 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:22.363243 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 374/738] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:08:22.363243 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364bb5a7-6725-409d-9fec-e231056f3bbb None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:22.366027 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d026d0-8d88-49a3-aa28-58039a75b0da None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:22.418513 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab7b4e-948c-4b39-a39e-9b91f6dbfd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:22.419657 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab7b4e-948c-4b39-a39e-9b91f6dbfd38 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:22.420160 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab7b4e-948c-4b39-a39e-9b91f6dbfd38 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:22.420669 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab7b4e-948c-4b39-a39e-9b91f6dbfd38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:22.463147 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364bb5a7-6725-409d-9fec-e231056f3bbb None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:22.497049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b859469-fd1f-45c4-b439-ea6e15f61b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:22.497842 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b859469-fd1f-45c4-b439-ea6e15f61b1f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:22.498129 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b859469-fd1f-45c4-b439-ea6e15f61b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:22.498407 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b859469-fd1f-45c4-b439-ea6e15f61b1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:22.499444 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab7b4e-948c-4b39-a39e-9b91f6dbfd38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:22.500909 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab7b4e-948c-4b39-a39e-9b91f6dbfd38 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:22.510680 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b859469-fd1f-45c4-b439-ea6e15f61b1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:22.512124 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b859469-fd1f-45c4-b439-ea6e15f61b1f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:22.538000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-7b859469-fd1f-45c4-b439-ea6e15f61b1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:22.538000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:22.538000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:22.538000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:22.538000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:22.538000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:22.538000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:22.538000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:22.538000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:22.538000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:22.538000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:22.538000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:22.538000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:22.538000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:22.538000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:22.538973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:22.538973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:22.538973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:22.538973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:22.538973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:22.538973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:22.538973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:22.538973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:22.538973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:22.538973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:22.538973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:22.538973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:22.538973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:22.538973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:22.538973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:22.538973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:22.538973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:22.539936 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:22.539936 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:22.539936 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:22.539936 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:22.539936 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 375/739] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:22.540463 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e8256-bed4-46ab-84e1-3217e81bf07e None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:22.540747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e8256-bed4-46ab-84e1-3217e81bf07e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:22.541027 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e8256-bed4-46ab-84e1-3217e81bf07e None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:22.543280 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 376/740] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:22.544022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e8256-bed4-46ab-84e1-3217e81bf07e None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:22.555441 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:22.579087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:22.606234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 365/741] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:08:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:08:22.607585 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959d981-5618-4c7f-983e-ae3eb3344162 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:22.607878 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959d981-5618-4c7f-983e-ae3eb3344162 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:22.608147 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959d981-5618-4c7f-983e-ae3eb3344162 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:22.611499 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 366/742] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:22.612156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f959d981-5618-4c7f-983e-ae3eb3344162 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:22.647827 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e8256-bed4-46ab-84e1-3217e81bf07e None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:22.681303 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d741445-afdc-4717-b641-24c5fb2417d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:22.681968 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d741445-afdc-4717-b641-24c5fb2417d6 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:22.682251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d741445-afdc-4717-b641-24c5fb2417d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:22.682526 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d741445-afdc-4717-b641-24c5fb2417d6 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:22.683263 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d741445-afdc-4717-b641-24c5fb2417d6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:22.684624 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d741445-afdc-4717-b641-24c5fb2417d6 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:22.699130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 377/743] 10.222.0.81 () {60 vars in 1141 bytes} [Tue Jun 7 14:08:22 2022] POST /identity/v3/services => generated 240 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:08:22.700420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7042226-706a-47bd-a484-af089a9ac987 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:22.700675 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7042226-706a-47bd-a484-af089a9ac987 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:22.700947 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7042226-706a-47bd-a484-af089a9ac987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:22.719834 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-c7042226-706a-47bd-a484-af089a9ac987 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:22.721077 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c7042226-706a-47bd-a484-af089a9ac987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:22.739572 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f959d981-5618-4c7f-983e-ae3eb3344162 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:22.781084 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52dc265-5313-402b-bc62-ccf2c54e43d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:22.781777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dc265-5313-402b-bc62-ccf2c54e43d8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:22.782062 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dc265-5313-402b-bc62-ccf2c54e43d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:22.782337 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dc265-5313-402b-bc62-ccf2c54e43d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:22.783081 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52dc265-5313-402b-bc62-ccf2c54e43d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:22.783420 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52dc265-5313-402b-bc62-ccf2c54e43d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:22.784823 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52dc265-5313-402b-bc62-ccf2c54e43d8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:22.796711 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:22.799465 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 367/744] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:22.800446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52dc265-5313-402b-bc62-ccf2c54e43d8 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:22.816043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 378/745] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:22 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:08:22.816043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a38a40-dc73-4f6e-b496-753f44fc3aba None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:22.816043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a38a40-dc73-4f6e-b496-753f44fc3aba None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:22.816043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a38a40-dc73-4f6e-b496-753f44fc3aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:22.831557 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-e4a38a40-dc73-4f6e-b496-753f44fc3aba None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:22.832962 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e4a38a40-dc73-4f6e-b496-753f44fc3aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:22.843169 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:22.919323 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 379/746] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:22 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:08:22.919323 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a38a40-dc73-4f6e-b496-753f44fc3aba None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:22.926549 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52dc265-5313-402b-bc62-ccf2c54e43d8 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:22.962490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0bfc6-0144-4d1c-bb78-e49cc5f76dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:22.963333 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0bfc6-0144-4d1c-bb78-e49cc5f76dff None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:22.963616 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0bfc6-0144-4d1c-bb78-e49cc5f76dff None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:22.964006 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0bfc6-0144-4d1c-bb78-e49cc5f76dff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:22.976167 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0bfc6-0144-4d1c-bb78-e49cc5f76dff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:22.977606 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0bfc6-0144-4d1c-bb78-e49cc5f76dff None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:22.996272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-40e0bfc6-0144-4d1c-bb78-e49cc5f76dff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:22.996272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:22.996272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:22.996272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:22.996272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:22.996272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:22.996272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:22.996272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:22.996272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:22.996272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:22.996272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:22.996272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:22.996272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:22.996272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:22.996272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:22.997202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:22.997202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:22.997202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:22.997202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:22.997202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:22.997202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:22.997202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:22.997202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:22.997202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:22.997202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:22.997202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:22.997202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:22.997202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:22.997202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:22.997202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:22.997202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:22.997202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:22.998703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:22.998703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:22.998703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:22.998703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:22.998703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 368/747] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:22.998703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0bfc6-0144-4d1c-bb78-e49cc5f76dff None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:23.033562 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a38a40-dc73-4f6e-b496-753f44fc3aba None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:23.067239 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:23.077544 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e81c9-024e-4fd4-b21e-275882e8a764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:23.078750 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e81c9-024e-4fd4-b21e-275882e8a764 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:23.079251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e81c9-024e-4fd4-b21e-275882e8a764 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:23.080113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e81c9-024e-4fd4-b21e-275882e8a764 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:23.087121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:23.089930 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e81c9-024e-4fd4-b21e-275882e8a764 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:23.091397 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e81c9-024e-4fd4-b21e-275882e8a764 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:23.105266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-af2e81c9-024e-4fd4-b21e-275882e8a764 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:23.105266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:23.105266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:23.105266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:23.105266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:23.105266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:23.105266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:23.105266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:23.105266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:23.105266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:23.105266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:23.105266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:23.105266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:08:23.105266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:08:23.105266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:08:23.106223 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:08:23.106223 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:23.106223 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:23.106223 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:23.106223 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:23.106223 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:23.106223 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:23.106223 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:23.106223 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:23.106223 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:23.106223 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:23.106223 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:23.106223 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:23.106223 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:23.106223 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:23.106223 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:23.106223 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:23.107151 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:08:23.107151 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:08:23.107151 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:08:23.107151 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:08:23.107151 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:08:23.107151 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:08:23.107151 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:23.107151 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:23.108281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0bfc6-0144-4d1c-bb78-e49cc5f76dff None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:23.111148 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 380/748] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:23.112130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e81c9-024e-4fd4-b21e-275882e8a764 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:23.147340 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320571c-4303-4cc2-9dda-3a3e716fb05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:23.148018 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320571c-4303-4cc2-9dda-3a3e716fb05b None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:23.148303 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320571c-4303-4cc2-9dda-3a3e716fb05b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:23.148578 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320571c-4303-4cc2-9dda-3a3e716fb05b None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:23.149301 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8320571c-4303-4cc2-9dda-3a3e716fb05b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:23.150685 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8320571c-4303-4cc2-9dda-3a3e716fb05b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:23.173349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 369/749] 10.222.0.81 () {60 vars in 1141 bytes} [Tue Jun 7 14:08:22 2022] POST /identity/v3/services => generated 251 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:08:23.173953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320571c-4303-4cc2-9dda-3a3e716fb05b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:23.252233 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e81c9-024e-4fd4-b21e-275882e8a764 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:23.300152 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320571c-4303-4cc2-9dda-3a3e716fb05b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:23.301013 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14adbf-4d29-4ae0-9d56-4e50198dc9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:23.302254 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14adbf-4d29-4ae0-9d56-4e50198dc9d9 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:23.302720 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14adbf-4d29-4ae0-9d56-4e50198dc9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:23.305347 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14adbf-4d29-4ae0-9d56-4e50198dc9d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:23.320859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d14adbf-4d29-4ae0-9d56-4e50198dc9d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:08:23.331250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-9d14adbf-4d29-4ae0-9d56-4e50198dc9d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:08:23.331250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:23.331250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:23.331250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:23.331250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:23.331250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:23.331250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:23.331250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:23.331250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:23.331250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:23.331250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:23.331250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:23.331250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:08:23.331250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:08:23.331250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:08:23.331250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:08:23.332266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:08:23.332266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:08:23.332266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:08:23.332266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:08:23.332266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:23.332266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:23.332266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:23.332266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:23.332266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:23.332266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:23.332266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:23.332266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:23.332266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:23.332266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:23.332266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:23.332266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:23.333122 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:23.333122 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:23.333122 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:23.333122 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:23.333122 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:08:23.333122 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:08:23.333122 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:08:23.333122 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:08:23.333122 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:08:23.333122 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:08:23.333122 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:08:23.333122 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:23.336660 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 381/750] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:08:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:23.338058 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14adbf-4d29-4ae0-9d56-4e50198dc9d9 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:23.344811 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50043ac5-9244-42c5-a956-2bd7e6f7ef04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:23.345371 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50043ac5-9244-42c5-a956-2bd7e6f7ef04 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:23.345756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50043ac5-9244-42c5-a956-2bd7e6f7ef04 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:23.346136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50043ac5-9244-42c5-a956-2bd7e6f7ef04 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:23.347030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50043ac5-9244-42c5-a956-2bd7e6f7ef04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:23.347527 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50043ac5-9244-42c5-a956-2bd7e6f7ef04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:23.349144 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50043ac5-9244-42c5-a956-2bd7e6f7ef04 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:23.358904 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:23.372813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 370/751] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:23.372813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50043ac5-9244-42c5-a956-2bd7e6f7ef04 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:23.414178 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:23.487721 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14adbf-4d29-4ae0-9d56-4e50198dc9d9 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:23.505165 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50043ac5-9244-42c5-a956-2bd7e6f7ef04 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:23.528194 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bbe53-1c77-4a52-8be0-d0f9434d98b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:23.529317 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bbe53-1c77-4a52-8be0-d0f9434d98b3 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:23.529888 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bbe53-1c77-4a52-8be0-d0f9434d98b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:23.530420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bbe53-1c77-4a52-8be0-d0f9434d98b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:23.531498 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212bbe53-1c77-4a52-8be0-d0f9434d98b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:23.532095 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212bbe53-1c77-4a52-8be0-d0f9434d98b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:23.533847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212bbe53-1c77-4a52-8be0-d0f9434d98b3 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:23.541181 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971441ed-f202-4d13-9c1a-791c40d1cd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:23.541864 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971441ed-f202-4d13-9c1a-791c40d1cd91 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:23.542151 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971441ed-f202-4d13-9c1a-791c40d1cd91 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:23.542431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971441ed-f202-4d13-9c1a-791c40d1cd91 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:23.543195 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971441ed-f202-4d13-9c1a-791c40d1cd91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:23.543536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971441ed-f202-4d13-9c1a-791c40d1cd91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:23.545287 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971441ed-f202-4d13-9c1a-791c40d1cd91 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:23.551116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 382/752] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:08:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:23.552323 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bbe53-1c77-4a52-8be0-d0f9434d98b3 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:23.559176 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 371/753] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:08:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:23.560242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971441ed-f202-4d13-9c1a-791c40d1cd91 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:23.624199 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:23.667537 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bbe53-1c77-4a52-8be0-d0f9434d98b3 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:23.669636 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971441ed-f202-4d13-9c1a-791c40d1cd91 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:23.678686 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:23.706616 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a611ed-0b29-4dff-999e-6ca0bfd7225f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:23.707305 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a611ed-0b29-4dff-999e-6ca0bfd7225f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:23.707632 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a611ed-0b29-4dff-999e-6ca0bfd7225f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:23.707867 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a611ed-0b29-4dff-999e-6ca0bfd7225f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:23.708631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2078cd-30a6-429a-976c-81ea772c53c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:23.708752 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a611ed-0b29-4dff-999e-6ca0bfd7225f None admin] RBAC: Adding query filter params (role.id=a72e5464b7b746978805326c224753bc, scope.project.id=f67bad575f8b4a47bee7c314497cb2fe, user.id=d00b8bc311234e6da47b365ac8a7183a) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:23.709032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a611ed-0b29-4dff-999e-6ca0bfd7225f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:23.709956 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2078cd-30a6-429a-976c-81ea772c53c9 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:23.710440 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2078cd-30a6-429a-976c-81ea772c53c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:23.710611 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a611ed-0b29-4dff-999e-6ca0bfd7225f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:23.711064 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2078cd-30a6-429a-976c-81ea772c53c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:23.716897 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 372/754] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:08:23 2022] GET /identity/v3/role_assignments?user.id=d00b8bc311234e6da47b365ac8a7183a&scope.project.id=f67bad575f8b4a47bee7c314497cb2fe&role.id=a72e5464b7b746978805326c224753bc => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:23.717855 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a611ed-0b29-4dff-999e-6ca0bfd7225f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:23.719244 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2078cd-30a6-429a-976c-81ea772c53c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:23.721049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2078cd-30a6-429a-976c-81ea772c53c9 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:23.737492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-3d2078cd-30a6-429a-976c-81ea772c53c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:23.737492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:23.737492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:23.737492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:23.737492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:23.737492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:23.737492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:23.737492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:23.737492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:23.737492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:23.737492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:23.737492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:23.737492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:08:23.737492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:08:23.737492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:08:23.738517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:08:23.738517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:23.738517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:23.738517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:23.738517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:23.738517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:23.738517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:23.738517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:23.738517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:23.738517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:23.738517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:23.738517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:23.738517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:23.738517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:23.738517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:23.738517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:23.738517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:23.739450 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:08:23.739450 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:08:23.739450 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:08:23.739450 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:08:23.739450 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:08:23.739450 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:08:23.739450 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:23.739450 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:23.742607 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 383/755] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:23.748901 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2078cd-30a6-429a-976c-81ea772c53c9 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:23.878854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a611ed-0b29-4dff-999e-6ca0bfd7225f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:23.879768 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2078cd-30a6-429a-976c-81ea772c53c9 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:23.923245 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:23.931122 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d56f22-bb9d-4617-bf14-177cac5c8325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:23.931845 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d56f22-bb9d-4617-bf14-177cac5c8325 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:23.932135 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d56f22-bb9d-4617-bf14-177cac5c8325 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:23.932417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d56f22-bb9d-4617-bf14-177cac5c8325 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:23.939252 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d56f22-bb9d-4617-bf14-177cac5c8325 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:23.939252 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d56f22-bb9d-4617-bf14-177cac5c8325 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:23.939252 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d56f22-bb9d-4617-bf14-177cac5c8325 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:23.945940 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278b7a8-ed83-4646-be4e-91a0c4d2f8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:23.947087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278b7a8-ed83-4646-be4e-91a0c4d2f8af None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:23.947588 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278b7a8-ed83-4646-be4e-91a0c4d2f8af None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:23.948058 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278b7a8-ed83-4646-be4e-91a0c4d2f8af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:23.953667 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 384/756] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:08:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:23.960274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d56f22-bb9d-4617-bf14-177cac5c8325 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:23.966464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278b7a8-ed83-4646-be4e-91a0c4d2f8af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:23.968584 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278b7a8-ed83-4646-be4e-91a0c4d2f8af None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:23.992682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:24.008361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-c278b7a8-ed83-4646-be4e-91a0c4d2f8af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:24.008361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:24.008361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:24.008361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:24.008361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:24.008361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:24.008361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:24.008361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:24.008361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:24.008361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:24.008361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:24.008361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:24.008361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:24.008361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:24.008361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:24.009406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:24.009406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:24.009406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:24.009406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:24.009406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:24.009406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:24.009406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:24.009406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:24.009406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:24.009406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:24.009406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:24.009406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:24.009406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:24.009406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:24.009406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:24.009406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:24.009406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:24.010286 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:24.010286 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:24.010286 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:24.010286 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:24.015664 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 373/757] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:24.021007 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278b7a8-ed83-4646-be4e-91a0c4d2f8af None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:24.069137 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d56f22-bb9d-4617-bf14-177cac5c8325 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:24.104336 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58957eb-7e9c-4725-902f-06daf4bcbb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:24.105016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58957eb-7e9c-4725-902f-06daf4bcbb2f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:24.105279 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58957eb-7e9c-4725-902f-06daf4bcbb2f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:24.105555 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58957eb-7e9c-4725-902f-06daf4bcbb2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:24.106387 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58957eb-7e9c-4725-902f-06daf4bcbb2f None admin] RBAC: Adding query filter params (role.id=82bbe3651b9b4f378fd150d93357b2b1, scope.project.id=f67bad575f8b4a47bee7c314497cb2fe, user.id=da7c0f99060a44eaab743ba71f159e88) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:24.106695 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58957eb-7e9c-4725-902f-06daf4bcbb2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:24.108150 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58957eb-7e9c-4725-902f-06daf4bcbb2f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:24.114184 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 385/758] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:08:23 2022] GET /identity/v3/role_assignments?user.id=da7c0f99060a44eaab743ba71f159e88&scope.project.id=f67bad575f8b4a47bee7c314497cb2fe&role.id=82bbe3651b9b4f378fd150d93357b2b1 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:24.239863 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278b7a8-ed83-4646-be4e-91a0c4d2f8af None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:24.289529 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29331a-d76d-4813-8f93-43b061b2321e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:24.290137 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29331a-d76d-4813-8f93-43b061b2321e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:24.290415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29331a-d76d-4813-8f93-43b061b2321e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:24.290687 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29331a-d76d-4813-8f93-43b061b2321e None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:24.291459 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29331a-d76d-4813-8f93-43b061b2321e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:24.291926 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29331a-d76d-4813-8f93-43b061b2321e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:24.293333 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29331a-d76d-4813-8f93-43b061b2321e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:24.311641 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 374/759] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:24.316633 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58957eb-7e9c-4725-902f-06daf4bcbb2f None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:24.453811 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:24.468104 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58957eb-7e9c-4725-902f-06daf4bcbb2f None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:24.513328 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90de22-fa33-4c3b-92ac-ae0c8ebff817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:24.514129 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90de22-fa33-4c3b-92ac-ae0c8ebff817 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:24.514413 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90de22-fa33-4c3b-92ac-ae0c8ebff817 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:24.514689 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90de22-fa33-4c3b-92ac-ae0c8ebff817 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:24.523164 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a90de22-fa33-4c3b-92ac-ae0c8ebff817 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:24.524758 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a90de22-fa33-4c3b-92ac-ae0c8ebff817 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:24.546063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-6a90de22-fa33-4c3b-92ac-ae0c8ebff817 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:08:24.546063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:24.546063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:24.546063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:24.546063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:24.546063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:24.546063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:24.546063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:24.546063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:24.546063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:24.546063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:24.546063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:24.546063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:08:24.546063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:08:24.546063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:08:24.546063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:08:24.548309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:24.548309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:24.548309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:08:24.548309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:24.548309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:08:24.548309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:24.548309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:24.548309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:24.548309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:24.548309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:24.548309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:24.548309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:24.548309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:24.548309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:24.548309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:24.548309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:24.548309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:24.548309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:24.550455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:24.550455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:24.550455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:08:24.550455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:08:24.550455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:08:24.550455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:08:24.550455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:08:24.550455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:08:24.550455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:08:24.550455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:24.550455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 386/760] 10.222.0.81 () {56 vars in 1093 bytes} [Tue Jun 7 14:08:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:24.556623 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29331a-d76d-4813-8f93-43b061b2321e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:24.629047 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:24.720570 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29331a-d76d-4813-8f93-43b061b2321e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:24.762785 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a221da4-e5ca-4e8e-9e20-05c73fc8acc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:24.763488 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a221da4-e5ca-4e8e-9e20-05c73fc8acc3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:24.763771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a221da4-e5ca-4e8e-9e20-05c73fc8acc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:24.764029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a221da4-e5ca-4e8e-9e20-05c73fc8acc3 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:24.764872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a221da4-e5ca-4e8e-9e20-05c73fc8acc3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:24.765246 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a221da4-e5ca-4e8e-9e20-05c73fc8acc3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:24.766585 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a221da4-e5ca-4e8e-9e20-05c73fc8acc3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:24.790349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 375/761] 10.222.0.81 () {56 vars in 1133 bytes} [Tue Jun 7 14:08:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:24.795766 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90de22-fa33-4c3b-92ac-ae0c8ebff817 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:24.972228 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90de22-fa33-4c3b-92ac-ae0c8ebff817 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:25.018728 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e316bf-144c-4461-8c65-c003153de447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:25.019631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e316bf-144c-4461-8c65-c003153de447 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:25.019825 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e316bf-144c-4461-8c65-c003153de447 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:25.020111 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e316bf-144c-4461-8c65-c003153de447 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:25.031276 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e316bf-144c-4461-8c65-c003153de447 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:25.036766 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e316bf-144c-4461-8c65-c003153de447 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:25.060203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-24e316bf-144c-4461-8c65-c003153de447 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:25.060203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:25.060203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:25.060203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:25.060203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:25.060203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:25.060203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:25.060203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:25.060203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:25.060203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:25.060203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:25.060203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:25.060203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:25.060203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:25.060203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:25.062251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:25.062251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:25.062251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:25.062251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:25.062251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:25.062251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:25.062251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:25.062251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:25.062251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:25.062251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:25.062251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:25.062251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:25.062251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:25.062251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:25.062251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:25.062251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:25.062251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:25.064490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:25.064490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:25.064490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:25.064490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:25.064490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 387/762] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:25.068319 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a221da4-e5ca-4e8e-9e20-05c73fc8acc3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:25.221838 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:25.233938 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a221da4-e5ca-4e8e-9e20-05c73fc8acc3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:25.282658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c03ee1-939e-49ef-90bc-9a7728e2686b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:25.283352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c03ee1-939e-49ef-90bc-9a7728e2686b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:25.283636 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c03ee1-939e-49ef-90bc-9a7728e2686b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:25.283905 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c03ee1-939e-49ef-90bc-9a7728e2686b None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:25.284629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c03ee1-939e-49ef-90bc-9a7728e2686b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:25.284929 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c03ee1-939e-49ef-90bc-9a7728e2686b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:25.286291 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c03ee1-939e-49ef-90bc-9a7728e2686b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:25.302539 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 376/763] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:25.308732 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e316bf-144c-4461-8c65-c003153de447 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:25.314722 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:25.462614 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797aa2ad-ec51-448f-8a9c-7ab99b0db25d None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:25.462966 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797aa2ad-ec51-448f-8a9c-7ab99b0db25d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:25.463169 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797aa2ad-ec51-448f-8a9c-7ab99b0db25d None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:25.464912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 377/764] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:25.469883 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e316bf-144c-4461-8c65-c003153de447 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:25.484791 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a2a76-c97b-4d19-b23b-7375fd06c5ec None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:25.485070 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a2a76-c97b-4d19-b23b-7375fd06c5ec None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:25.485252 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a2a76-c97b-4d19-b23b-7375fd06c5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:25.505735 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26edb002-e527-4a72-9eb0-f36f4a8f1edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:25.507717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edb002-e527-4a72-9eb0-f36f4a8f1edb None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:25.508239 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edb002-e527-4a72-9eb0-f36f4a8f1edb None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:25.508608 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edb002-e527-4a72-9eb0-f36f4a8f1edb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:25.515933 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26edb002-e527-4a72-9eb0-f36f4a8f1edb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:25.517383 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26edb002-e527-4a72-9eb0-f36f4a8f1edb None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:25.520512 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2d4a2a76-c97b-4d19-b23b-7375fd06c5ec None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:25.522353 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2d4a2a76-c97b-4d19-b23b-7375fd06c5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:25.533646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-26edb002-e527-4a72-9eb0-f36f4a8f1edb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:25.533646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:25.533646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:25.533646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:25.533646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:25.533646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:25.533646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:25.533646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:25.533646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:25.533646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:25.533646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:25.533646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:25.533646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:08:25.533646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:08:25.533646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:08:25.535737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:08:25.535737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:25.535737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:25.535737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:25.535737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:25.535737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:25.535737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:25.535737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:25.535737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:25.535737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:25.535737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:25.535737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:25.535737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:25.535737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:25.535737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:25.535737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:25.535737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:25.538435 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:08:25.538435 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:08:25.538435 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:08:25.538435 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:08:25.538435 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:08:25.538435 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:08:25.538435 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:25.538435 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:25.538435 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 388/765] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:25.545824 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26edb002-e527-4a72-9eb0-f36f4a8f1edb None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:25.651499 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 378/766] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:08:25.671572 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:25.677429 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a2a76-c97b-4d19-b23b-7375fd06c5ec None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:25.755582 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26edb002-e527-4a72-9eb0-f36f4a8f1edb None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:25.816381 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02421300-f817-4abe-a07e-aa19f7fc9dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:25.817169 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02421300-f817-4abe-a07e-aa19f7fc9dad None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:25.817357 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02421300-f817-4abe-a07e-aa19f7fc9dad None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:25.817631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02421300-f817-4abe-a07e-aa19f7fc9dad None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:25.818499 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02421300-f817-4abe-a07e-aa19f7fc9dad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:25.818935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02421300-f817-4abe-a07e-aa19f7fc9dad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:25.827242 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02421300-f817-4abe-a07e-aa19f7fc9dad None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:25.839498 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 389/767] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:08:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:25.842047 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:25.845959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02421300-f817-4abe-a07e-aa19f7fc9dad None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:25.873417 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a2a76-c97b-4d19-b23b-7375fd06c5ec None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:25.935211 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79d728-032a-4b38-b881-ebfd3497f4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:25.936045 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79d728-032a-4b38-b881-ebfd3497f4ff None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:25.936324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79d728-032a-4b38-b881-ebfd3497f4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:25.936595 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79d728-032a-4b38-b881-ebfd3497f4ff None admin] PATH_INFO: `/v3/services/network` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:25.937355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff79d728-032a-4b38-b881-ebfd3497f4ff None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:25.938913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff79d728-032a-4b38-b881-ebfd3497f4ff None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:25.957437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-ff79d728-032a-4b38-b881-ebfd3497f4ff None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 14:08:25.957437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:25.957437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:08:25.957437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:08:25.957437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:08:25.957437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:08:25.957437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:08:25.957437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:25.957437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 14:08:25.957437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:25.957437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:08:25.957437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:25.957437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:25.957437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:25.957437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:25.957437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:25.957437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:25.958498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:25.958498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:25.958498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:25.958498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:25.958498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:25.958498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:25.958498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:08:25.958498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:08:25.958498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:08:25.958498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:08:25.958498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:25.958498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:25.958498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:25.958498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:25.958498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:25.958498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:25.959701 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:25.959701 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:25.959701 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:25.959701 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:25.959701 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:25.959701 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:25.959701 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:25.959701 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:25.959701 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:25.959701 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:25.959701 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:08:25.959701 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:25.959701 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 14:08:25.959701 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:25.964023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 379/768] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:25 2022] GET /identity/v3/services/network => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:08:25.966993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79d728-032a-4b38-b881-ebfd3497f4ff None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:26.024253 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02421300-f817-4abe-a07e-aa19f7fc9dad None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:26.070655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:26.089557 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2a4a5-0ba6-445d-b1f9-9c584ca1bbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:26.090456 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2a4a5-0ba6-445d-b1f9-9c584ca1bbd5 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:26.090739 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2a4a5-0ba6-445d-b1f9-9c584ca1bbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:26.091016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2a4a5-0ba6-445d-b1f9-9c584ca1bbd5 None admin] PATH_INFO: `/v3/projects/f67bad575f8b4a47bee7c314497cb2fe/users/2ae726fbc9cf40e68c7587552a118a7a/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:26.094349 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2a4a5-0ba6-445d-b1f9-9c584ca1bbd5 None admin] RBAC: Authorizing `identity:create_grant(project_id=f67bad575f8b4a47bee7c314497cb2fe, user_id=2ae726fbc9cf40e68c7587552a118a7a, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:26.097209 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2a4a5-0ba6-445d-b1f9-9c584ca1bbd5 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:26.114637 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 390/769] 10.222.0.81 () {58 vars in 1439 bytes} [Tue Jun 7 14:08:25 2022] PUT /identity/v3/projects/f67bad575f8b4a47bee7c314497cb2fe/users/2ae726fbc9cf40e68c7587552a118a7a/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:08:26.124327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe3fde-66b6-431f-8169-e1559bdc14cb None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:26.124650 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe3fde-66b6-431f-8169-e1559bdc14cb None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:26.124895 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe3fde-66b6-431f-8169-e1559bdc14cb None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:26.126615 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 391/770] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:26 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:26.132745 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da96ea-65dd-4668-a935-8904c40a25e3 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:26.133064 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da96ea-65dd-4668-a935-8904c40a25e3 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:26.133302 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da96ea-65dd-4668-a935-8904c40a25e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:26.153528 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79d728-032a-4b38-b881-ebfd3497f4ff None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:26.176414 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-27da96ea-65dd-4668-a935-8904c40a25e3 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:26.178201 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-27da96ea-65dd-4668-a935-8904c40a25e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:26.194138 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d9966-79c4-455a-a763-09a87272ee7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:26.194836 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d9966-79c4-455a-a763-09a87272ee7c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:26.195117 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d9966-79c4-455a-a763-09a87272ee7c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:26.195466 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d9966-79c4-455a-a763-09a87272ee7c None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:26.196198 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d9966-79c4-455a-a763-09a87272ee7c None admin] RBAC: Adding query filter params (name=network) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:26.196619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d9966-79c4-455a-a763-09a87272ee7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:26.198008 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d9966-79c4-455a-a763-09a87272ee7c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:26.203722 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 380/771] 10.222.0.81 () {56 vars in 1112 bytes} [Tue Jun 7 14:08:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:26.209593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d9966-79c4-455a-a763-09a87272ee7c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:26.227701 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:26.318134 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 392/772] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:08:26.339938 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d9966-79c4-455a-a763-09a87272ee7c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:26.356942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f590d71-500b-42d7-8653-ad83e2edd1bc None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:26.357578 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f590d71-500b-42d7-8653-ad83e2edd1bc None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:26.358089 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f590d71-500b-42d7-8653-ad83e2edd1bc None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:26.360032 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 393/773] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:08:26.366691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70802a-fdd1-4050-baef-683d095d3b37 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:26.367364 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70802a-fdd1-4050-baef-683d095d3b37 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:26.367846 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70802a-fdd1-4050-baef-683d095d3b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:26.386625 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-0e70802a-fdd1-4050-baef-683d095d3b37 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:26.388394 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0e70802a-fdd1-4050-baef-683d095d3b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:26.393081 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255493b-52db-4f4d-867a-75fe51ee5b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:26.393767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255493b-52db-4f4d-867a-75fe51ee5b4a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:26.394046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255493b-52db-4f4d-867a-75fe51ee5b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:26.394318 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255493b-52db-4f4d-867a-75fe51ee5b4a None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:26.395040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255493b-52db-4f4d-867a-75fe51ee5b4a None admin] RBAC: Adding query filter params (type=network) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:26.395435 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255493b-52db-4f4d-867a-75fe51ee5b4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:26.396985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255493b-52db-4f4d-867a-75fe51ee5b4a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:26.407418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 381/774] 10.222.0.81 () {56 vars in 1112 bytes} [Tue Jun 7 14:08:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:26.407418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255493b-52db-4f4d-867a-75fe51ee5b4a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:26.494020 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 394/775] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:08:26.499979 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70802a-fdd1-4050-baef-683d095d3b37 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:26.524269 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:26.554673 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255493b-52db-4f4d-867a-75fe51ee5b4a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:26.575452 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:26.598801 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c027352-f457-47e4-b63e-928a7bcacc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:26.599931 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c027352-f457-47e4-b63e-928a7bcacc40 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:26.600235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c027352-f457-47e4-b63e-928a7bcacc40 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:26.600503 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c027352-f457-47e4-b63e-928a7bcacc40 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:26.601541 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c027352-f457-47e4-b63e-928a7bcacc40 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:26.603177 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c027352-f457-47e4-b63e-928a7bcacc40 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:26.618818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-5c027352-f457-47e4-b63e-928a7bcacc40 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 14:08:26.618818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:26.618818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:08:26.618818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:08:26.618818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:08:26.618818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:08:26.618818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:08:26.618818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:26.618818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 14:08:26.618818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:26.618818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:08:26.618818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:26.618818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:26.618818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:26.618818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:26.618818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:26.619831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:26.619831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:26.619831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:26.619831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:26.619831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:26.619831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:26.619831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:26.619831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:08:26.619831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:08:26.619831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:08:26.619831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:08:26.619831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:26.619831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:26.619831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:26.619831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:26.619831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:26.620965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:26.620965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:26.620965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:26.620965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:26.620965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:26.620965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:26.620965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:26.620965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:26.620965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:26.620965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:26.620965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:26.620965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:08:26.620965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:26.620965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 14:08:26.620965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:26.620965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 382/776] 10.222.0.81 () {56 vars in 1129 bytes} [Tue Jun 7 14:08:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:26.621809 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c027352-f457-47e4-b63e-928a7bcacc40 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:26.656353 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70802a-fdd1-4050-baef-683d095d3b37 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:26.704168 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd0e51-b29f-4e2f-ad5c-18887a4c2220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:26.705728 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd0e51-b29f-4e2f-ad5c-18887a4c2220 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:26.706218 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd0e51-b29f-4e2f-ad5c-18887a4c2220 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:26.707171 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd0e51-b29f-4e2f-ad5c-18887a4c2220 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:26.708597 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd0e51-b29f-4e2f-ad5c-18887a4c2220 None admin] RBAC: Adding query filter params (interface=public, service_id=69ca1c897e084be493d87442b0d6268a, region_id=RegionOne) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:26.710136 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd0e51-b29f-4e2f-ad5c-18887a4c2220 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:26.712019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd0e51-b29f-4e2f-ad5c-18887a4c2220 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:26.724614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 395/777] 10.222.0.81 () {56 vars in 1251 bytes} [Tue Jun 7 14:08:26 2022] GET /identity/v3/endpoints?service_id=69ca1c897e084be493d87442b0d6268a&interface=public®ion_id=RegionOne => generated 194 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:26.726456 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd0e51-b29f-4e2f-ad5c-18887a4c2220 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:26.731598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c027352-f457-47e4-b63e-928a7bcacc40 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:26.771737 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d4df2-4ec1-4373-a574-ddb35b43d3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:26.772554 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d4df2-4ec1-4373-a574-ddb35b43d3a6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:26.773115 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d4df2-4ec1-4373-a574-ddb35b43d3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:26.773250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d4df2-4ec1-4373-a574-ddb35b43d3a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:26.782320 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:26.785739 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64d4df2-4ec1-4373-a574-ddb35b43d3a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:08:26.795049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-e64d4df2-4ec1-4373-a574-ddb35b43d3a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:26.795049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:26.795049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:26.795049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:26.795049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:26.795049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:26.795049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:26.795049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:26.795049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:26.795049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:26.795049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:26.795049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:26.795049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:08:26.795049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:08:26.795049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:08:26.795049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:08:26.796063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:08:26.796063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:08:26.796063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:08:26.796063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:08:26.796063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:26.796063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:26.796063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:26.796063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:26.796063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:26.796063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:26.796063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:26.796063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:26.796063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:26.796063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:26.796063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:26.796063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:26.796917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:26.796917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:26.796917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:26.796917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:26.796917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:08:26.796917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:08:26.796917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:08:26.796917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:08:26.796917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:08:26.796917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:08:26.796917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:26.796917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:26.801187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 383/778] 10.222.0.81 () {56 vars in 1102 bytes} [Tue Jun 7 14:08:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:26.802045 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d4df2-4ec1-4373-a574-ddb35b43d3a6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:26.857417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd0e51-b29f-4e2f-ad5c-18887a4c2220 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:26.867726 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:26.897303 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d393a3-8ec0-49db-8364-5140db8954a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:26.898195 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d393a3-8ec0-49db-8364-5140db8954a0 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:26.898318 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d393a3-8ec0-49db-8364-5140db8954a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:26.898524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d393a3-8ec0-49db-8364-5140db8954a0 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:26.899253 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d393a3-8ec0-49db-8364-5140db8954a0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:26.899563 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d393a3-8ec0-49db-8364-5140db8954a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:26.900938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d393a3-8ec0-49db-8364-5140db8954a0 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:26.905642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 396/779] 10.222.0.81 () {56 vars in 1124 bytes} [Tue Jun 7 14:08:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:26.906381 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d393a3-8ec0-49db-8364-5140db8954a0 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:26.920730 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d4df2-4ec1-4373-a574-ddb35b43d3a6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:26.956708 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c22386-c3e6-428e-85b8-413f37dd4f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:26.957454 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c22386-c3e6-428e-85b8-413f37dd4f4b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:26.957738 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c22386-c3e6-428e-85b8-413f37dd4f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:26.957943 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c22386-c3e6-428e-85b8-413f37dd4f4b None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:26.958636 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c22386-c3e6-428e-85b8-413f37dd4f4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:26.960755 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c22386-c3e6-428e-85b8-413f37dd4f4b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:26.971610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 384/780] 10.222.0.81 () {56 vars in 1087 bytes} [Tue Jun 7 14:08:26 2022] GET /identity/v3/services => generated 772 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:26.973343 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c91060-23b7-481a-869b-b6a9a8a08c18 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:26.973638 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c91060-23b7-481a-869b-b6a9a8a08c18 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:26.973923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c91060-23b7-481a-869b-b6a9a8a08c18 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:26.982105 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 385/781] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:26 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:26.983690 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c91060-23b7-481a-869b-b6a9a8a08c18 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:27.033108 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d393a3-8ec0-49db-8364-5140db8954a0 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:27.072968 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967c68f-4033-445a-8fea-2b5e364b8a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:27.074058 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967c68f-4033-445a-8fea-2b5e364b8a74 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:27.074553 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967c68f-4033-445a-8fea-2b5e364b8a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:27.075034 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967c68f-4033-445a-8fea-2b5e364b8a74 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:27.076085 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7967c68f-4033-445a-8fea-2b5e364b8a74 None admin] RBAC: Adding query filter params (name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:27.076619 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7967c68f-4033-445a-8fea-2b5e364b8a74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:27.078247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7967c68f-4033-445a-8fea-2b5e364b8a74 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:27.083505 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:27.089715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 397/782] 10.222.0.81 () {56 vars in 1103 bytes} [Tue Jun 7 14:08:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:27.091485 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930cb68-5951-4b5b-b527-8572497b9ab9 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:27.091990 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930cb68-5951-4b5b-b527-8572497b9ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:27.092463 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930cb68-5951-4b5b-b527-8572497b9ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:27.117655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-0930cb68-5951-4b5b-b527-8572497b9ab9 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:27.118800 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0930cb68-5951-4b5b-b527-8572497b9ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:27.137768 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c91060-23b7-481a-869b-b6a9a8a08c18 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:27.167059 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:27.187673 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dddc6f5-bcd3-4dea-91d6-46f4b6e8b108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:27.188356 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddc6f5-bcd3-4dea-91d6-46f4b6e8b108 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:27.189219 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddc6f5-bcd3-4dea-91d6-46f4b6e8b108 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:27.189219 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddc6f5-bcd3-4dea-91d6-46f4b6e8b108 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:27.189891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dddc6f5-bcd3-4dea-91d6-46f4b6e8b108 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:27.189891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dddc6f5-bcd3-4dea-91d6-46f4b6e8b108 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:27.191983 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dddc6f5-bcd3-4dea-91d6-46f4b6e8b108 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:27.199164 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 386/783] 10.222.0.81 () {56 vars in 1124 bytes} [Tue Jun 7 14:08:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:27.201303 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dddc6f5-bcd3-4dea-91d6-46f4b6e8b108 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:27.226625 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 398/784] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:08:27.227573 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930cb68-5951-4b5b-b527-8572497b9ab9 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:27.335017 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dddc6f5-bcd3-4dea-91d6-46f4b6e8b108 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:27.363529 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:27.364906 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930cb68-5951-4b5b-b527-8572497b9ab9 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:27.375212 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b9155-81a2-48ca-bee4-756a688e2272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:27.376006 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b9155-81a2-48ca-bee4-756a688e2272 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:27.376294 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b9155-81a2-48ca-bee4-756a688e2272 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:27.376586 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b9155-81a2-48ca-bee4-756a688e2272 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:27.389023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b9155-81a2-48ca-bee4-756a688e2272 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:27.390488 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b9155-81a2-48ca-bee4-756a688e2272 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:27.410520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-a18b9155-81a2-48ca-bee4-756a688e2272 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:27.410520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:27.410520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:27.410520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:27.410520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:27.410520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:27.410520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:27.410520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:27.410520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:27.410520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:27.410520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:27.410520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:27.410520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:27.410520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:27.410520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:27.411700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:27.411700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:27.411700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:27.411700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:27.411700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:27.411700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:27.411700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:27.411700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:27.411700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:27.411700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:27.411700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:27.411700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:27.411700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:27.411700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:27.411700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:27.411700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:27.411700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:27.413057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:27.413057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:27.413057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:27.413057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:27.413057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 387/785] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:27.413057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b9155-81a2-48ca-bee4-756a688e2272 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:27.414255 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc159f-206f-4675-822e-9eff74fc2a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:27.415086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc159f-206f-4675-822e-9eff74fc2a69 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:27.415828 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc159f-206f-4675-822e-9eff74fc2a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:27.416308 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc159f-206f-4675-822e-9eff74fc2a69 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:27.418331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc159f-206f-4675-822e-9eff74fc2a69 None admin] RBAC: Adding query filter params (interface=public, service_id=c4cee0c64c544679b68d9156144bdaad, region_id=RegionOne) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:27.418844 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc159f-206f-4675-822e-9eff74fc2a69 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:27.420435 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc159f-206f-4675-822e-9eff74fc2a69 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:27.428455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 399/786] 10.222.0.81 () {56 vars in 1251 bytes} [Tue Jun 7 14:08:27 2022] GET /identity/v3/endpoints?service_id=c4cee0c64c544679b68d9156144bdaad&interface=public®ion_id=RegionOne => generated 194 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:27.429946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc159f-206f-4675-822e-9eff74fc2a69 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:27.465844 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:27.540176 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b9155-81a2-48ca-bee4-756a688e2272 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:27.564877 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc159f-206f-4675-822e-9eff74fc2a69 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:27.577081 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69ee84-d1e9-4038-ad6f-dd61b926fc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:27.577883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69ee84-d1e9-4038-ad6f-dd61b926fc73 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:27.578166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69ee84-d1e9-4038-ad6f-dd61b926fc73 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:27.578442 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69ee84-d1e9-4038-ad6f-dd61b926fc73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:27.594373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d69ee84-d1e9-4038-ad6f-dd61b926fc73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:08:27.602773 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-5d69ee84-d1e9-4038-ad6f-dd61b926fc73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:08:27.602773 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:27.602773 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:27.602773 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:27.602773 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:27.602773 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:27.602773 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:27.602773 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:27.602773 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:27.602773 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:27.602773 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:27.602773 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:27.602773 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:08:27.602773 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:08:27.602773 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:08:27.602773 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:08:27.603778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:08:27.603778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:08:27.603778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:08:27.603778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:08:27.603778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:27.603778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:27.603778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:27.603778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:27.603778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:27.603778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:27.603778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:27.603778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:27.603778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:27.603778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:27.603778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:27.603778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:27.604635 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:27.604635 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:27.604635 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:27.604635 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:27.604635 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:08:27.604635 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:08:27.604635 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:08:27.604635 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:08:27.604635 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:08:27.604635 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:08:27.604635 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:08:27.604635 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:27.609015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 388/787] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:08:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:27.609978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69ee84-d1e9-4038-ad6f-dd61b926fc73 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:27.612856 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ac8e7-45a8-4c50-8a82-719d629e4f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:27.613933 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ac8e7-45a8-4c50-8a82-719d629e4f8a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:27.614544 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ac8e7-45a8-4c50-8a82-719d629e4f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:27.615025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ac8e7-45a8-4c50-8a82-719d629e4f8a None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:27.616921 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ac8e7-45a8-4c50-8a82-719d629e4f8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:27.617444 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ac8e7-45a8-4c50-8a82-719d629e4f8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:27.619085 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ac8e7-45a8-4c50-8a82-719d629e4f8a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:27.637579 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 400/788] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:27.638928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ac8e7-45a8-4c50-8a82-719d629e4f8a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:27.656435 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:27.719660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69ee84-d1e9-4038-ad6f-dd61b926fc73 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:27.735539 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:27.749854 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ac8e7-45a8-4c50-8a82-719d629e4f8a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:27.758220 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81952c0-7914-460d-a23e-94def1bc9cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:27.759195 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81952c0-7914-460d-a23e-94def1bc9cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:27.759340 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81952c0-7914-460d-a23e-94def1bc9cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:27.759714 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81952c0-7914-460d-a23e-94def1bc9cf1 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:27.760403 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81952c0-7914-460d-a23e-94def1bc9cf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:27.761930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81952c0-7914-460d-a23e-94def1bc9cf1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:27.767665 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 389/789] 10.222.0.81 () {56 vars in 1087 bytes} [Tue Jun 7 14:08:27 2022] GET /identity/v3/services => generated 772 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:27.768279 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81952c0-7914-460d-a23e-94def1bc9cf1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:27.791095 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d102e2-b9fe-4ea7-ad4d-cdf8cbd94a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:27.792206 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d102e2-b9fe-4ea7-ad4d-cdf8cbd94a5e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:27.792719 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d102e2-b9fe-4ea7-ad4d-cdf8cbd94a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:27.793201 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d102e2-b9fe-4ea7-ad4d-cdf8cbd94a5e None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:27.794841 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d102e2-b9fe-4ea7-ad4d-cdf8cbd94a5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:27.795410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d102e2-b9fe-4ea7-ad4d-cdf8cbd94a5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:27.797508 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d102e2-b9fe-4ea7-ad4d-cdf8cbd94a5e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:27.809586 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 401/790] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:08:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:27.817153 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d102e2-b9fe-4ea7-ad4d-cdf8cbd94a5e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:27.986322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81952c0-7914-460d-a23e-94def1bc9cf1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:27.988923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:28.033128 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d102e2-b9fe-4ea7-ad4d-cdf8cbd94a5e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:28.044875 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2bbf56-a124-42e1-8c9b-ad6f379b6356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:28.045706 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bbf56-a124-42e1-8c9b-ad6f379b6356 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:28.045989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bbf56-a124-42e1-8c9b-ad6f379b6356 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:28.046261 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bbf56-a124-42e1-8c9b-ad6f379b6356 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:28.055643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2bbf56-a124-42e1-8c9b-ad6f379b6356 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:28.057085 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2bbf56-a124-42e1-8c9b-ad6f379b6356 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:28.073059 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5bfa4-2db8-440c-b559-0ab3bec09500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:28.074163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5bfa4-2db8-440c-b559-0ab3bec09500 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:28.074651 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5bfa4-2db8-440c-b559-0ab3bec09500 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:28.075130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5bfa4-2db8-440c-b559-0ab3bec09500 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:28.075285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-0d2bbf56-a124-42e1-8c9b-ad6f379b6356 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 14:08:28.075285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:28.075285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:28.075285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:28.075285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:28.075285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:28.075285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:28.075285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:28.075285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:28.075285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:28.075285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:28.075285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:28.075285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:08:28.075285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:08:28.075285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:08:28.075285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:08:28.076166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:28.076166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:28.076166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:08:28.076166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:28.076166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:08:28.076166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:28.076166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:28.076166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:28.076166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:28.076166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:28.076166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:28.076166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:28.076166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:28.076166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:28.076166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:28.076166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:28.076166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:28.076166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:28.077125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:28.077125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:28.077125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:08:28.077125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:08:28.077125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:08:28.077125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:08:28.077125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:08:28.077125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:08:28.077125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 14:08:28.077125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:28.077125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 390/791] 10.222.0.81 () {56 vars in 1117 bytes} [Tue Jun 7 14:08:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:28.082287 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2bbf56-a124-42e1-8c9b-ad6f379b6356 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:28.092364 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5bfa4-2db8-440c-b559-0ab3bec09500 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:28.094268 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5bfa4-2db8-440c-b559-0ab3bec09500 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:28.114227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-a1a5bfa4-2db8-440c-b559-0ab3bec09500 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:28.114227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:28.114227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:28.114227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:28.114227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:28.114227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:28.114227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:28.114227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:28.114227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:28.114227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:28.114227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:28.114227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:28.114227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:28.114227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:28.114227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:28.115363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:28.115363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:28.115363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:28.115363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:28.115363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:28.115363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:28.115363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:28.115363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:28.115363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:28.115363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:28.115363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:28.115363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:28.115363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:28.115363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:28.115363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:28.115363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:28.115363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:28.116316 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:28.116316 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:28.116316 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:28.116316 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:28.118673 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 402/792] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:28.128842 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5bfa4-2db8-440c-b559-0ab3bec09500 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:28.132584 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:28.209541 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2bbf56-a124-42e1-8c9b-ad6f379b6356 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:28.263488 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5bfa4-2db8-440c-b559-0ab3bec09500 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:28.282320 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c14d7-0f03-4955-89e6-a3920cb52e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:28.283024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c14d7-0f03-4955-89e6-a3920cb52e0c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:28.283335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c14d7-0f03-4955-89e6-a3920cb52e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:28.283617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c14d7-0f03-4955-89e6-a3920cb52e0c None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:28.284493 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c14d7-0f03-4955-89e6-a3920cb52e0c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:28.284807 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c14d7-0f03-4955-89e6-a3920cb52e0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:28.286299 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c14d7-0f03-4955-89e6-a3920cb52e0c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:28.329142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 391/793] 10.222.0.81 () {56 vars in 1149 bytes} [Tue Jun 7 14:08:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:28.341530 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c14d7-0f03-4955-89e6-a3920cb52e0c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:28.346500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f9930-c794-48da-a2f1-e8aa60e88953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:28.347245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f9930-c794-48da-a2f1-e8aa60e88953 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:28.347528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f9930-c794-48da-a2f1-e8aa60e88953 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:28.347808 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f9930-c794-48da-a2f1-e8aa60e88953 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:28.348546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72f9930-c794-48da-a2f1-e8aa60e88953 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:28.348854 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72f9930-c794-48da-a2f1-e8aa60e88953 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:28.350781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72f9930-c794-48da-a2f1-e8aa60e88953 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:28.371420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 403/794] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:28.378246 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f9930-c794-48da-a2f1-e8aa60e88953 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:28.389189 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:28.495722 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:28.532366 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c14d7-0f03-4955-89e6-a3920cb52e0c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:28.558015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f9930-c794-48da-a2f1-e8aa60e88953 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:28.595828 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c314f7-9b57-420e-955c-a83a633b7a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:28.596631 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c314f7-9b57-420e-955c-a83a633b7a43 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:28.596915 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c314f7-9b57-420e-955c-a83a633b7a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:28.597192 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c314f7-9b57-420e-955c-a83a633b7a43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:28.612155 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c314f7-9b57-420e-955c-a83a633b7a43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:28.613621 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c314f7-9b57-420e-955c-a83a633b7a43 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:28.619497 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a26d8-5c96-4855-a7c7-872f642e3e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:28.620680 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a26d8-5c96-4855-a7c7-872f642e3e10 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:28.620966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a26d8-5c96-4855-a7c7-872f642e3e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:28.621239 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a26d8-5c96-4855-a7c7-872f642e3e10 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:28.634690 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a26d8-5c96-4855-a7c7-872f642e3e10 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:28.636235 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a26d8-5c96-4855-a7c7-872f642e3e10 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:28.641414 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-65c314f7-9b57-420e-955c-a83a633b7a43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:28.641414 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:28.641414 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:28.641414 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:28.641414 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:28.641414 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:28.641414 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:28.641414 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:28.641414 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:28.641414 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:28.641414 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:28.641414 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:28.641414 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:28.641414 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:28.641414 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:28.642311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:28.642311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:28.642311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:28.642311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:28.642311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:28.642311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:28.642311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:28.642311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:28.642311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:28.642311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:28.642311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:28.642311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:28.642311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:28.642311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:28.642311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:28.642311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:28.642311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:28.643195 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:28.643195 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:28.643195 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:28.643195 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:28.643195 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 392/795] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:08:28.643909 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d338e0f-2f7e-46b6-b536-a84a60b83a1e None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:28.644192 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d338e0f-2f7e-46b6-b536-a84a60b83a1e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:28.644530 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d338e0f-2f7e-46b6-b536-a84a60b83a1e None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:28.649994 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 393/796] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:28.650779 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d338e0f-2f7e-46b6-b536-a84a60b83a1e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:28.651749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-f31a26d8-5c96-4855-a7c7-872f642e3e10 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 14:08:28.651749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:28.651749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:28.651749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:28.651749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:28.651749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:28.651749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:28.651749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:28.651749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:28.651749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:28.651749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:28.651749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:28.651749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:08:28.651749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:08:28.651749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:08:28.651749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:08:28.652677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:28.652677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:28.652677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:08:28.652677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:28.652677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:08:28.652677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:28.652677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:28.652677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:28.652677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:28.652677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:28.652677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:28.652677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:28.652677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:28.652677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:28.652677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:28.652677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:28.652677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:28.652677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:28.653607 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:28.653607 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:28.653607 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:08:28.653607 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:08:28.653607 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:08:28.653607 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:08:28.653607 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:08:28.653607 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:08:28.653607 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 14:08:28.653607 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:28.653607 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 404/797] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:08:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:08:28.655557 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64889828-b135-43a7-b3b1-12574172f90f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:28.655557 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64889828-b135-43a7-b3b1-12574172f90f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:28.655557 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64889828-b135-43a7-b3b1-12574172f90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:28.677057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-64889828-b135-43a7-b3b1-12574172f90f None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:28.679059 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-64889828-b135-43a7-b3b1-12574172f90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:28.741107 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:28.753235 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 405/798] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:08:28.754473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64889828-b135-43a7-b3b1-12574172f90f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:28.770458 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d338e0f-2f7e-46b6-b536-a84a60b83a1e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:28.810446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b485169-4231-4877-8e9b-f79689937d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:28.811030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b485169-4231-4877-8e9b-f79689937d12 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:28.811339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b485169-4231-4877-8e9b-f79689937d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:28.811614 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b485169-4231-4877-8e9b-f79689937d12 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:28.812394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b485169-4231-4877-8e9b-f79689937d12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:28.812702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b485169-4231-4877-8e9b-f79689937d12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:28.814019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:28.815396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b485169-4231-4877-8e9b-f79689937d12 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:28.829306 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 394/799] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:28.830396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b485169-4231-4877-8e9b-f79689937d12 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:28.872546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64889828-b135-43a7-b3b1-12574172f90f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:28.910066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c270f52-9ed2-4812-bec2-a7c7fbfa2b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:28.910860 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c270f52-9ed2-4812-bec2-a7c7fbfa2b7a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:28.911234 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c270f52-9ed2-4812-bec2-a7c7fbfa2b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:28.911593 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c270f52-9ed2-4812-bec2-a7c7fbfa2b7a None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:28.913100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c270f52-9ed2-4812-bec2-a7c7fbfa2b7a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:28.913581 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c270f52-9ed2-4812-bec2-a7c7fbfa2b7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:28.915111 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c270f52-9ed2-4812-bec2-a7c7fbfa2b7a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:28.939497 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 406/800] 10.222.0.81 () {56 vars in 1135 bytes} [Tue Jun 7 14:08:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:28.939497 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c270f52-9ed2-4812-bec2-a7c7fbfa2b7a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:28.948888 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b485169-4231-4877-8e9b-f79689937d12 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:28.986745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8ca68-a76d-4a93-9c1d-977d05014304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:28.988014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8ca68-a76d-4a93-9c1d-977d05014304 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:28.988475 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8ca68-a76d-4a93-9c1d-977d05014304 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:28.988921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8ca68-a76d-4a93-9c1d-977d05014304 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:28.992229 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:29.003078 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d8ca68-a76d-4a93-9c1d-977d05014304 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:08:29.012134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-e7d8ca68-a76d-4a93-9c1d-977d05014304 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:08:29.012134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:29.012134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:29.012134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:29.012134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:29.012134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:29.012134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:29.012134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:29.012134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:29.012134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:29.012134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:29.012134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:29.012134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:08:29.012134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:08:29.012134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:08:29.012134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:08:29.013079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:08:29.013079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:08:29.013079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:08:29.013079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:08:29.013079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:29.013079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:29.013079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:29.013079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:29.013079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:29.013079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:29.013079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:29.013079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:29.013079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:29.013079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:29.013079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:29.013079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:29.013923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:29.013923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:29.013923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:29.013923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:29.013923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:08:29.013923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:08:29.013923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:08:29.013923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:08:29.013923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:08:29.013923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:08:29.013923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:08:29.013923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:29.016635 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 395/801] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:08:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:29.017615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8ca68-a76d-4a93-9c1d-977d05014304 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:29.049724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c270f52-9ed2-4812-bec2-a7c7fbfa2b7a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:29.062481 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:29.088636 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59692ac-ee60-40d9-83c3-593a4d0d261f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:29.089545 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59692ac-ee60-40d9-83c3-593a4d0d261f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:29.089903 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59692ac-ee60-40d9-83c3-593a4d0d261f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:29.090340 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59692ac-ee60-40d9-83c3-593a4d0d261f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:29.097931 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59692ac-ee60-40d9-83c3-593a4d0d261f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:29.099888 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59692ac-ee60-40d9-83c3-593a4d0d261f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:29.116105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-a59692ac-ee60-40d9-83c3-593a4d0d261f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:29.116105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:29.116105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:29.116105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:29.116105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:29.116105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:29.116105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:29.116105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:29.116105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:29.116105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:29.116105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:29.116105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:29.116105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:08:29.116105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:08:29.116105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:08:29.117262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:08:29.117262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:29.117262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:29.117262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:29.117262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:29.117262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:29.117262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:29.117262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:29.117262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:29.117262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:29.117262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:29.117262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:29.117262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:29.117262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:29.117262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:29.117262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:29.117262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:29.118161 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:08:29.118161 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:08:29.118161 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:08:29.118161 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:08:29.118161 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:08:29.118161 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:08:29.118161 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:29.118161 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:29.122496 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 407/802] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:29.123446 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59692ac-ee60-40d9-83c3-593a4d0d261f None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:29.138801 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8ca68-a76d-4a93-9c1d-977d05014304 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:29.180859 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dcc99-893c-4f91-a7ee-52944c7a6185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:29.181906 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dcc99-893c-4f91-a7ee-52944c7a6185 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:29.182353 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dcc99-893c-4f91-a7ee-52944c7a6185 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:29.182723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dcc99-893c-4f91-a7ee-52944c7a6185 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:29.194969 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704dcc99-893c-4f91-a7ee-52944c7a6185 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:29.196569 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704dcc99-893c-4f91-a7ee-52944c7a6185 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:29.215629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-704dcc99-893c-4f91-a7ee-52944c7a6185 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:29.215629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:29.215629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:29.215629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:29.215629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:29.215629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:29.215629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:29.215629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:29.215629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:29.215629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:29.215629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:29.215629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:29.215629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:29.215629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:29.215629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:29.216981 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:29.216981 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:29.216981 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:29.216981 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:29.216981 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:29.216981 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:29.216981 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:29.216981 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:29.216981 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:29.216981 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:29.216981 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:29.216981 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:29.216981 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:29.216981 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:29.216981 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:29.216981 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:29.216981 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:29.218303 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:29.218303 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:29.218303 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:29.218303 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:29.221881 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 396/803] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:29.222902 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dcc99-893c-4f91-a7ee-52944c7a6185 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:29.238686 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59692ac-ee60-40d9-83c3-593a4d0d261f None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:29.277551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:29.278891 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999958c-a64e-435c-b653-ed8c5a90423d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:29.279680 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999958c-a64e-435c-b653-ed8c5a90423d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:29.279972 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999958c-a64e-435c-b653-ed8c5a90423d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:29.280305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999958c-a64e-435c-b653-ed8c5a90423d None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:29.281077 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5999958c-a64e-435c-b653-ed8c5a90423d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:29.282325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5999958c-a64e-435c-b653-ed8c5a90423d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:29.283754 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5999958c-a64e-435c-b653-ed8c5a90423d None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:29.293792 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 408/804] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:08:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:29.294704 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999958c-a64e-435c-b653-ed8c5a90423d None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:29.326862 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:29.344474 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dcc99-893c-4f91-a7ee-52944c7a6185 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:29.383423 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40f4fa-1f1f-4b65-9ea9-95f39df00c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:29.384337 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40f4fa-1f1f-4b65-9ea9-95f39df00c7c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:29.384795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40f4fa-1f1f-4b65-9ea9-95f39df00c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:29.385332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40f4fa-1f1f-4b65-9ea9-95f39df00c7c None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:29.386372 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed40f4fa-1f1f-4b65-9ea9-95f39df00c7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:29.386825 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed40f4fa-1f1f-4b65-9ea9-95f39df00c7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:29.388447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed40f4fa-1f1f-4b65-9ea9-95f39df00c7c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:29.402126 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999958c-a64e-435c-b653-ed8c5a90423d None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:29.404415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 397/805] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:08:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:29.405519 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40f4fa-1f1f-4b65-9ea9-95f39df00c7c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:29.442989 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee201e-3fb3-4769-baab-9afa39cc4a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:29.444421 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee201e-3fb3-4769-baab-9afa39cc4a72 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:29.445420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee201e-3fb3-4769-baab-9afa39cc4a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:29.445420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee201e-3fb3-4769-baab-9afa39cc4a72 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:29.445712 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee201e-3fb3-4769-baab-9afa39cc4a72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:29.446019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee201e-3fb3-4769-baab-9afa39cc4a72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:29.447505 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee201e-3fb3-4769-baab-9afa39cc4a72 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:29.460875 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 409/806] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:29.461799 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee201e-3fb3-4769-baab-9afa39cc4a72 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:29.516689 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40f4fa-1f1f-4b65-9ea9-95f39df00c7c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:29.525384 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:29.558819 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:29.561311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1032d7-15a0-4cf7-9caa-88e5edf05085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:29.562906 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1032d7-15a0-4cf7-9caa-88e5edf05085 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:29.563741 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1032d7-15a0-4cf7-9caa-88e5edf05085 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:29.564321 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1032d7-15a0-4cf7-9caa-88e5edf05085 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:29.572210 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee201e-3fb3-4769-baab-9afa39cc4a72 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:29.583289 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1032d7-15a0-4cf7-9caa-88e5edf05085 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:29.586584 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1032d7-15a0-4cf7-9caa-88e5edf05085 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:29.607803 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-ee1032d7-15a0-4cf7-9caa-88e5edf05085 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:29.607803 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:29.607803 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:29.607803 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:29.607803 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:29.607803 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:29.607803 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:29.607803 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:29.607803 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:29.607803 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:29.607803 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:29.607803 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:29.607803 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:29.607803 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:29.607803 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:29.608756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:29.608756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:29.608756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:29.608756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:29.608756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:29.608756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:29.608756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:29.608756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:29.608756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:29.608756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:29.608756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:29.608756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:29.608756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:29.608756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:29.608756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:29.608756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:29.608756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:29.609663 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:29.609663 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:29.609663 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:29.609663 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:29.609663 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 398/807] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:29.610657 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1032d7-15a0-4cf7-9caa-88e5edf05085 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:29.617450 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f4c60-897e-4e74-b1d5-134d4bc18990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:29.618355 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f4c60-897e-4e74-b1d5-134d4bc18990 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:29.618639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f4c60-897e-4e74-b1d5-134d4bc18990 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:29.618913 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f4c60-897e-4e74-b1d5-134d4bc18990 None admin] PATH_INFO: `/v3/projects/f67bad575f8b4a47bee7c314497cb2fe/users/d00b8bc311234e6da47b365ac8a7183a/roles/a72e5464b7b746978805326c224753bc` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:29.634953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f4c60-897e-4e74-b1d5-134d4bc18990 None admin] RBAC: Authorizing `identity:create_grant(project_id=f67bad575f8b4a47bee7c314497cb2fe, user_id=d00b8bc311234e6da47b365ac8a7183a, role_id=a72e5464b7b746978805326c224753bc)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:29.636489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f4c60-897e-4e74-b1d5-134d4bc18990 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:29.650476 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 410/808] 10.222.0.81 () {58 vars in 1439 bytes} [Tue Jun 7 14:08:29 2022] PUT /identity/v3/projects/f67bad575f8b4a47bee7c314497cb2fe/users/d00b8bc311234e6da47b365ac8a7183a/roles/a72e5464b7b746978805326c224753bc => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:08:29.652154 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7102dde-c7e9-43a4-9e53-0e9661a753f4 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:29.652655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7102dde-c7e9-43a4-9e53-0e9661a753f4 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:29.653337 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7102dde-c7e9-43a4-9e53-0e9661a753f4 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:29.654908 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 411/809] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:29.655942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7102dde-c7e9-43a4-9e53-0e9661a753f4 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:29.735682 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1032d7-15a0-4cf7-9caa-88e5edf05085 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:29.776926 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f85626-c5f2-4257-87b9-c892454bd6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:29.778189 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f85626-c5f2-4257-87b9-c892454bd6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:29.778683 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f85626-c5f2-4257-87b9-c892454bd6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:29.779163 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f85626-c5f2-4257-87b9-c892454bd6c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:29.788157 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f85626-c5f2-4257-87b9-c892454bd6c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:29.789979 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f85626-c5f2-4257-87b9-c892454bd6c9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:29.805529 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-08f85626-c5f2-4257-87b9-c892454bd6c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:29.805529 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:29.805529 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:29.805529 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:29.805529 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:29.805529 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:29.805529 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:29.805529 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:29.805529 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:29.805529 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:29.805529 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:29.805529 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:29.805529 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:08:29.805529 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:08:29.805529 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:08:29.806514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:08:29.806514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:29.806514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:29.806514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:29.806514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:29.806514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:29.806514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:29.806514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:29.806514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:29.806514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:29.806514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:29.806514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:29.806514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:29.806514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:29.806514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:29.806514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:29.806514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:29.807877 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:08:29.807877 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:08:29.807877 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:08:29.807877 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:08:29.807877 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:08:29.807877 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:08:29.807877 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:29.807877 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:29.810214 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 399/810] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:29.811900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60c25f-5c65-4746-bd77-600147b42db8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:29.812397 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60c25f-5c65-4746-bd77-600147b42db8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:29.812878 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60c25f-5c65-4746-bd77-600147b42db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:29.821708 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7102dde-c7e9-43a4-9e53-0e9661a753f4 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:29.831922 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0f60c25f-5c65-4746-bd77-600147b42db8 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:29.833577 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0f60c25f-5c65-4746-bd77-600147b42db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:29.850540 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:29.872279 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e186c-2fe1-4cdd-8488-1aff7a94c339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:29.872279 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e186c-2fe1-4cdd-8488-1aff7a94c339 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:29.872583 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e186c-2fe1-4cdd-8488-1aff7a94c339 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:29.872838 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e186c-2fe1-4cdd-8488-1aff7a94c339 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:29.874106 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e186c-2fe1-4cdd-8488-1aff7a94c339 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:29.874106 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e186c-2fe1-4cdd-8488-1aff7a94c339 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:29.875314 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e186c-2fe1-4cdd-8488-1aff7a94c339 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:29.887849 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 412/811] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:29.888544 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e186c-2fe1-4cdd-8488-1aff7a94c339 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:29.898293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:29.942148 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 400/812] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:08:29.943256 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f60c25f-5c65-4746-bd77-600147b42db8 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:30.000052 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e186c-2fe1-4cdd-8488-1aff7a94c339 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:30.037863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973d951-fe15-4588-bf91-4ddc30bfe66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:30.038532 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973d951-fe15-4588-bf91-4ddc30bfe66d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:30.038821 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973d951-fe15-4588-bf91-4ddc30bfe66d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:30.039166 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973d951-fe15-4588-bf91-4ddc30bfe66d None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:30.040055 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9973d951-fe15-4588-bf91-4ddc30bfe66d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:30.040411 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9973d951-fe15-4588-bf91-4ddc30bfe66d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:30.042077 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9973d951-fe15-4588-bf91-4ddc30bfe66d None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:30.049890 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f60c25f-5c65-4746-bd77-600147b42db8 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:30.057767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 413/813] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:08:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:30.058749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973d951-fe15-4588-bf91-4ddc30bfe66d None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:30.094946 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa837d-cba2-413d-a456-5f34c4849a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:30.095854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa837d-cba2-413d-a456-5f34c4849a7e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:30.096146 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa837d-cba2-413d-a456-5f34c4849a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:30.096575 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa837d-cba2-413d-a456-5f34c4849a7e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:30.108416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa837d-cba2-413d-a456-5f34c4849a7e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:30.110222 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa837d-cba2-413d-a456-5f34c4849a7e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:30.127650 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:30.129794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-8efa837d-cba2-413d-a456-5f34c4849a7e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:08:30.129794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:30.129794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:30.129794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:30.129794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:30.129794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:30.129794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:30.129794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:30.129794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:30.129794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:30.129794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:30.129794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:30.129794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:08:30.129794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:08:30.129794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:08:30.129794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:08:30.131548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:30.131548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:30.131548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:08:30.131548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:30.131548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:08:30.131548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:30.131548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:30.131548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:30.131548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:30.131548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:30.131548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:30.131548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:30.131548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:30.131548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:30.131548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:30.131548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:30.131548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:30.131548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:30.132692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:30.132692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:30.132692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:08:30.132692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:08:30.132692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:08:30.132692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:08:30.132692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:08:30.132692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:08:30.132692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:08:30.132692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:30.132692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 401/814] 10.222.0.81 () {56 vars in 1093 bytes} [Tue Jun 7 14:08:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:30.133451 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa837d-cba2-413d-a456-5f34c4849a7e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:30.148582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:30.184037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973d951-fe15-4588-bf91-4ddc30bfe66d None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:30.222641 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b02bc-ab9b-4600-9445-e879023561a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:30.223765 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b02bc-ab9b-4600-9445-e879023561a9 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:30.224237 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b02bc-ab9b-4600-9445-e879023561a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:30.224884 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b02bc-ab9b-4600-9445-e879023561a9 None admin] PATH_INFO: `/v3/services/network` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:30.225863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b02bc-ab9b-4600-9445-e879023561a9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:30.227514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b02bc-ab9b-4600-9445-e879023561a9 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:30.241143 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-e72b02bc-ab9b-4600-9445-e879023561a9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 14:08:30.241143 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:30.241143 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:08:30.241143 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:08:30.241143 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:08:30.241143 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:08:30.241143 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:08:30.241143 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:30.241143 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 14:08:30.241143 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:30.241143 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:08:30.241143 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:30.241143 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:30.241143 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:30.241143 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:30.241143 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:30.241143 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:30.242203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:30.242203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:30.242203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:30.242203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:30.242203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:30.242203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:30.242203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:08:30.242203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:08:30.242203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:08:30.242203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:08:30.242203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:30.242203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:30.242203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:30.242203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:30.242203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:30.242203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:30.243066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:30.243066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:30.243066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:30.243066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:30.243066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:30.243066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:30.243066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:30.243066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:30.243066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:30.243066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:30.243066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:08:30.243066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:30.243066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 14:08:30.243066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:30.246010 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 414/815] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:30 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:30.246989 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b02bc-ab9b-4600-9445-e879023561a9 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:30.259322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa837d-cba2-413d-a456-5f34c4849a7e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:30.296281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0966c-abef-4cc3-a7f7-e4ffa535e9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:30.297288 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0966c-abef-4cc3-a7f7-e4ffa535e9dd None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:30.297642 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0966c-abef-4cc3-a7f7-e4ffa535e9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:30.297989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0966c-abef-4cc3-a7f7-e4ffa535e9dd None admin] PATH_INFO: `/v3/projects/f67bad575f8b4a47bee7c314497cb2fe/users/da7c0f99060a44eaab743ba71f159e88/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:30.300144 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d0966c-abef-4cc3-a7f7-e4ffa535e9dd None admin] RBAC: Authorizing `identity:create_grant(project_id=f67bad575f8b4a47bee7c314497cb2fe, user_id=da7c0f99060a44eaab743ba71f159e88, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:30.301727 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d0966c-abef-4cc3-a7f7-e4ffa535e9dd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:30.319304 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 402/816] 10.222.0.81 () {58 vars in 1439 bytes} [Tue Jun 7 14:08:30 2022] PUT /identity/v3/projects/f67bad575f8b4a47bee7c314497cb2fe/users/da7c0f99060a44eaab743ba71f159e88/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:08:30.320442 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0966c-abef-4cc3-a7f7-e4ffa535e9dd None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:30.423606 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b02bc-ab9b-4600-9445-e879023561a9 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:30.464931 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:30.493196 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:30.513411 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9414b8a-2741-4dc5-a5ee-39f753f0c27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:30.515365 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9414b8a-2741-4dc5-a5ee-39f753f0c27e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:30.515852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9414b8a-2741-4dc5-a5ee-39f753f0c27e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:30.516336 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9414b8a-2741-4dc5-a5ee-39f753f0c27e None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:30.517399 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9414b8a-2741-4dc5-a5ee-39f753f0c27e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:30.517905 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9414b8a-2741-4dc5-a5ee-39f753f0c27e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:30.518479 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0966c-abef-4cc3-a7f7-e4ffa535e9dd None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:30.519875 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9414b8a-2741-4dc5-a5ee-39f753f0c27e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:30.543315 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 415/817] 10.222.0.81 () {56 vars in 1133 bytes} [Tue Jun 7 14:08:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:30.545942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d2203-6191-4189-9d30-2f093bb1a6fe None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:30.546442 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d2203-6191-4189-9d30-2f093bb1a6fe None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:30.547235 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d2203-6191-4189-9d30-2f093bb1a6fe None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:30.549950 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 416/818] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:30.551044 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d2203-6191-4189-9d30-2f093bb1a6fe None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:30.562713 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943cca9-272c-4f2d-8693-c842cc76c5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:30.563515 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943cca9-272c-4f2d-8693-c842cc76c5a4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:30.563869 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943cca9-272c-4f2d-8693-c842cc76c5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:30.564155 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943cca9-272c-4f2d-8693-c842cc76c5a4 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:30.564875 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b943cca9-272c-4f2d-8693-c842cc76c5a4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:30.565179 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b943cca9-272c-4f2d-8693-c842cc76c5a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:30.566566 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b943cca9-272c-4f2d-8693-c842cc76c5a4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:30.573141 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 403/819] 10.222.0.81 () {56 vars in 1112 bytes} [Tue Jun 7 14:08:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:30.574417 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2efb8-d6ac-47f1-80cb-4db86589d4ea None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:30.574702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2efb8-d6ac-47f1-80cb-4db86589d4ea None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:30.574982 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2efb8-d6ac-47f1-80cb-4db86589d4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:30.593431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d4b2efb8-d6ac-47f1-80cb-4db86589d4ea None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:30.594634 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d4b2efb8-d6ac-47f1-80cb-4db86589d4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:30.688061 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 404/820] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:08:30.689293 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2efb8-d6ac-47f1-80cb-4db86589d4ea None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:30.718042 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d2203-6191-4189-9d30-2f093bb1a6fe None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:30.762617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:30.766726 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fed80-2f9c-467b-a30e-87b88c9b816e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:30.767703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fed80-2f9c-467b-a30e-87b88c9b816e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:30.768193 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fed80-2f9c-467b-a30e-87b88c9b816e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:30.768694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fed80-2f9c-467b-a30e-87b88c9b816e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:30.813581 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2efb8-d6ac-47f1-80cb-4db86589d4ea None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:30.838910 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:30.861699 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96344b1-e072-436f-838f-619285cc3a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:30.862598 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6fed80-2f9c-467b-a30e-87b88c9b816e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:30.862944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96344b1-e072-436f-838f-619285cc3a12 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:30.863474 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96344b1-e072-436f-838f-619285cc3a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:30.863967 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96344b1-e072-436f-838f-619285cc3a12 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:30.864238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6fed80-2f9c-467b-a30e-87b88c9b816e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:30.865648 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96344b1-e072-436f-838f-619285cc3a12 None admin] RBAC: Adding query filter params (type=network) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:30.866157 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96344b1-e072-436f-838f-619285cc3a12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:30.867782 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96344b1-e072-436f-838f-619285cc3a12 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:30.878391 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 405/821] 10.222.0.81 () {56 vars in 1112 bytes} [Tue Jun 7 14:08:30 2022] GET /identity/v3/services?type=network => generated 350 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:30.879578 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96344b1-e072-436f-838f-619285cc3a12 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:31.002882 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 417/822] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:08:30 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:08:31.003892 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fed80-2f9c-467b-a30e-87b88c9b816e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:31.004899 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96344b1-e072-436f-838f-619285cc3a12 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:31.040912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:31.053433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd4bca9-0b5b-432a-a23c-df4b049ae6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:31.055255 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4bca9-0b5b-432a-a23c-df4b049ae6ea None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:31.055688 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4bca9-0b5b-432a-a23c-df4b049ae6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:31.056132 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4bca9-0b5b-432a-a23c-df4b049ae6ea None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:31.057060 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd4bca9-0b5b-432a-a23c-df4b049ae6ea None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:31.058839 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd4bca9-0b5b-432a-a23c-df4b049ae6ea None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:31.075745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-2dd4bca9-0b5b-432a-a23c-df4b049ae6ea None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 14:08:31.075745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:31.075745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:08:31.075745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:08:31.075745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:08:31.075745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:08:31.075745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:08:31.075745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:31.075745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 14:08:31.075745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:31.075745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:08:31.075745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:31.075745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:31.075745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:31.075745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:31.075745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:31.076917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:31.076917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:31.076917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:31.076917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:31.076917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:31.076917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:31.076917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:31.076917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:08:31.076917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:08:31.076917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:08:31.076917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:08:31.076917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:31.076917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:31.076917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:31.076917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:31.076917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:31.077962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:31.077962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:31.077962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:31.077962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:31.077962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:31.077962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:31.077962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:31.077962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:31.077962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:31.077962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:31.077962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:31.077962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:08:31.077962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:31.077962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 14:08:31.077962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:31.081590 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 406/823] 10.222.0.81 () {56 vars in 1129 bytes} [Tue Jun 7 14:08:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:31.082594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd4bca9-0b5b-432a-a23c-df4b049ae6ea None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:31.095010 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:31.114348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fed80-2f9c-467b-a30e-87b88c9b816e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:31.158609 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b884b83-8375-41e6-b17e-eb0e982886c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:31.159344 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b884b83-8375-41e6-b17e-eb0e982886c6 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:31.159624 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b884b83-8375-41e6-b17e-eb0e982886c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:31.159902 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b884b83-8375-41e6-b17e-eb0e982886c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:31.160641 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b884b83-8375-41e6-b17e-eb0e982886c6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:31.165021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b884b83-8375-41e6-b17e-eb0e982886c6 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:31.190519 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 418/824] 10.222.0.81 () {60 vars in 1144 bytes} [Tue Jun 7 14:08:31 2022] POST /identity/v3/endpoints => generated 327 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:08:31.191584 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b884b83-8375-41e6-b17e-eb0e982886c6 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:31.223563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd4bca9-0b5b-432a-a23c-df4b049ae6ea None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:31.269733 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfa788-7143-4d17-a98c-24fb37dba432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:31.271929 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfa788-7143-4d17-a98c-24fb37dba432 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:31.272612 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfa788-7143-4d17-a98c-24fb37dba432 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:31.273186 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfa788-7143-4d17-a98c-24fb37dba432 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:31.286748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfa788-7143-4d17-a98c-24fb37dba432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:31.289668 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfa788-7143-4d17-a98c-24fb37dba432 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:31.309475 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-abcfa788-7143-4d17-a98c-24fb37dba432 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:31.309475 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:31.309475 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:31.309475 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:31.309475 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:31.309475 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:31.309475 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:31.309475 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:31.309475 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:31.309475 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:31.309475 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:31.309475 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:31.309475 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:31.309475 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:31.309475 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:31.310491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:31.310491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:31.310491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:31.310491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:31.310491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:31.310491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:31.310491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:31.310491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:31.310491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:31.310491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:31.310491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:31.310491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:31.310491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:31.310491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:31.310491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:31.310491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:31.310491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:31.311465 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:31.311465 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:31.311465 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:31.311465 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:31.314052 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 407/825] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:31.321295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfa788-7143-4d17-a98c-24fb37dba432 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:31.340414 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b884b83-8375-41e6-b17e-eb0e982886c6 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:31.403607 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:31.417001 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8bca45-a503-4557-9aec-de7a8e69cc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:31.417686 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bca45-a503-4557-9aec-de7a8e69cc0b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:31.417967 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bca45-a503-4557-9aec-de7a8e69cc0b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:31.418239 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bca45-a503-4557-9aec-de7a8e69cc0b None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:31.418956 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8bca45-a503-4557-9aec-de7a8e69cc0b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:31.419278 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8bca45-a503-4557-9aec-de7a8e69cc0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:31.420669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8bca45-a503-4557-9aec-de7a8e69cc0b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:31.436960 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 419/826] 10.222.0.81 () {56 vars in 1124 bytes} [Tue Jun 7 14:08:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:31.442850 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8bca45-a503-4557-9aec-de7a8e69cc0b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:31.450434 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:31.518843 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfa788-7143-4d17-a98c-24fb37dba432 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:31.559197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b53aa6-b100-495f-bd60-b3589c2aec25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:31.559930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b53aa6-b100-495f-bd60-b3589c2aec25 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:31.560216 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b53aa6-b100-495f-bd60-b3589c2aec25 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:31.560585 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b53aa6-b100-495f-bd60-b3589c2aec25 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:31.561315 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b53aa6-b100-495f-bd60-b3589c2aec25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:31.561616 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b53aa6-b100-495f-bd60-b3589c2aec25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:31.562985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b53aa6-b100-495f-bd60-b3589c2aec25 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:31.577776 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 408/827] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:31.579181 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8bca45-a503-4557-9aec-de7a8e69cc0b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:31.583413 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b53aa6-b100-495f-bd60-b3589c2aec25 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:31.621546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be22a14-3b9a-443b-829b-6c980b6c36e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:31.622210 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be22a14-3b9a-443b-829b-6c980b6c36e0 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:31.622492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be22a14-3b9a-443b-829b-6c980b6c36e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:31.622765 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be22a14-3b9a-443b-829b-6c980b6c36e0 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:31.623492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be22a14-3b9a-443b-829b-6c980b6c36e0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:31.623795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be22a14-3b9a-443b-829b-6c980b6c36e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:31.625372 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be22a14-3b9a-443b-829b-6c980b6c36e0 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:31.631902 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 420/828] 10.222.0.81 () {56 vars in 1124 bytes} [Tue Jun 7 14:08:31 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:31.637448 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be22a14-3b9a-443b-829b-6c980b6c36e0 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:31.713006 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:31.718746 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b53aa6-b100-495f-bd60-b3589c2aec25 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:31.731270 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:31.766146 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475e012-b5c6-404c-8173-ca0ca1e87a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:31.767201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475e012-b5c6-404c-8173-ca0ca1e87a48 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:31.767920 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475e012-b5c6-404c-8173-ca0ca1e87a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:31.768204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475e012-b5c6-404c-8173-ca0ca1e87a48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:31.774269 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be22a14-3b9a-443b-829b-6c980b6c36e0 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:31.778648 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475e012-b5c6-404c-8173-ca0ca1e87a48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:31.780158 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475e012-b5c6-404c-8173-ca0ca1e87a48 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:31.799242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-7475e012-b5c6-404c-8173-ca0ca1e87a48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:31.799242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:31.799242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:31.799242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:31.799242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:31.799242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:31.799242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:31.799242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:31.799242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:31.799242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:31.799242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:31.799242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:31.799242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:08:31.799242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:08:31.799242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:08:31.800197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:08:31.800197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:31.800197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:31.800197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:31.800197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:31.800197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:31.800197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:31.800197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:31.800197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:31.800197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:31.800197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:31.800197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:31.800197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:31.800197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:31.800197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:31.800197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:31.800197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:31.801099 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:08:31.801099 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:08:31.801099 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:08:31.801099 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:08:31.801099 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:08:31.801099 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:08:31.801099 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:31.801099 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:31.801640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 409/829] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:31.806818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475e012-b5c6-404c-8173-ca0ca1e87a48 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:31.813349 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4c1a5-2cd6-4132-9b79-9443c49782d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:31.813950 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4c1a5-2cd6-4132-9b79-9443c49782d2 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:31.814228 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4c1a5-2cd6-4132-9b79-9443c49782d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:31.814499 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4c1a5-2cd6-4132-9b79-9443c49782d2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:31.815257 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc4c1a5-2cd6-4132-9b79-9443c49782d2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:31.816738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc4c1a5-2cd6-4132-9b79-9443c49782d2 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:31.842518 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 421/830] 10.222.0.81 () {60 vars in 1144 bytes} [Tue Jun 7 14:08:31 2022] POST /identity/v3/endpoints => generated 346 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:08:31.937141 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475e012-b5c6-404c-8173-ca0ca1e87a48 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:31.981978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5c62a-aecc-4411-aa26-344db9c1c445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:31.982656 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5c62a-aecc-4411-aa26-344db9c1c445 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:31.982935 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5c62a-aecc-4411-aa26-344db9c1c445 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:31.983208 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5c62a-aecc-4411-aa26-344db9c1c445 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:31.984035 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e5c62a-aecc-4411-aa26-344db9c1c445 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:31.984402 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e5c62a-aecc-4411-aa26-344db9c1c445 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:31.985920 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e5c62a-aecc-4411-aa26-344db9c1c445 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:32.001485 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 410/831] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:08:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:32.009370 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4c1a5-2cd6-4132-9b79-9443c49782d2 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:32.118227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4c1a5-2cd6-4132-9b79-9443c49782d2 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:32.144092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:32.160910 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c085e-9587-49d1-8c67-6f4e14db8f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:32.164307 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c085e-9587-49d1-8c67-6f4e14db8f8b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:32.164307 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c085e-9587-49d1-8c67-6f4e14db8f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:32.164307 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c085e-9587-49d1-8c67-6f4e14db8f8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:32.164307 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c085e-9587-49d1-8c67-6f4e14db8f8b None admin] RBAC: Adding query filter params (role.id=82bbe3651b9b4f378fd150d93357b2b1, scope.project.id=f67bad575f8b4a47bee7c314497cb2fe, user.id=2ae726fbc9cf40e68c7587552a118a7a) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:32.164307 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c085e-9587-49d1-8c67-6f4e14db8f8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:32.166592 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c085e-9587-49d1-8c67-6f4e14db8f8b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:32.171712 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 422/832] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:08:32 2022] GET /identity/v3/role_assignments?user.id=2ae726fbc9cf40e68c7587552a118a7a&scope.project.id=f67bad575f8b4a47bee7c314497cb2fe&role.id=82bbe3651b9b4f378fd150d93357b2b1 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:32.220320 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873290a-7dfd-482e-b09c-907521ab4d48 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:32.220605 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873290a-7dfd-482e-b09c-907521ab4d48 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:32.220886 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873290a-7dfd-482e-b09c-907521ab4d48 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:32.222577 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 411/833] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:32.228975 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889b986-1b0c-4dd2-b5f5-f7f7bdc13783 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:32.229261 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889b986-1b0c-4dd2-b5f5-f7f7bdc13783 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:32.229536 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889b986-1b0c-4dd2-b5f5-f7f7bdc13783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:32.248586 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-b889b986-1b0c-4dd2-b5f5-f7f7bdc13783 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:32.249825 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b889b986-1b0c-4dd2-b5f5-f7f7bdc13783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:32.324312 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 423/834] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:08:32.347919 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6873290a-7dfd-482e-b09c-907521ab4d48 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:32.365851 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:32.475253 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6873290a-7dfd-482e-b09c-907521ab4d48 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:32.510885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225b550-847e-45b5-981d-2c3d0042bb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:32.511727 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225b550-847e-45b5-981d-2c3d0042bb4b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:32.512012 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225b550-847e-45b5-981d-2c3d0042bb4b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:32.512284 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225b550-847e-45b5-981d-2c3d0042bb4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:32.525087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225b550-847e-45b5-981d-2c3d0042bb4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:08:32.533281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-c225b550-847e-45b5-981d-2c3d0042bb4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:32.533281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:32.533281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:32.533281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:32.533281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:32.533281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:32.533281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:32.533281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:32.533281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:32.533281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:32.533281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:32.533281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:32.533281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:08:32.533281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:08:32.533281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:08:32.533281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:08:32.534381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:08:32.534381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:08:32.534381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:08:32.534381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:08:32.534381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:32.534381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:32.534381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:32.534381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:32.534381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:32.534381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:32.534381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:32.534381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:32.534381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:32.534381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:32.534381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:32.534381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:32.535325 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:32.535325 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:32.535325 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:32.535325 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:32.535325 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:08:32.535325 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:08:32.535325 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:08:32.535325 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:08:32.535325 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:08:32.535325 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:08:32.535325 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:08:32.535325 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:32.535325 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 412/835] 10.222.0.81 () {56 vars in 1102 bytes} [Tue Jun 7 14:08:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:32.541206 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889b986-1b0c-4dd2-b5f5-f7f7bdc13783 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:32.645403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889b986-1b0c-4dd2-b5f5-f7f7bdc13783 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:32.664401 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:32.681900 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d6a15-688b-4c38-9092-5fe456c0a382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:32.682581 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d6a15-688b-4c38-9092-5fe456c0a382 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:32.682860 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d6a15-688b-4c38-9092-5fe456c0a382 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:32.683134 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d6a15-688b-4c38-9092-5fe456c0a382 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:32.683923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d6a15-688b-4c38-9092-5fe456c0a382 None admin] RBAC: Adding query filter params (name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:32.684229 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d6a15-688b-4c38-9092-5fe456c0a382 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:32.685598 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d6a15-688b-4c38-9092-5fe456c0a382 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:32.699291 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 424/836] 10.222.0.81 () {56 vars in 1103 bytes} [Tue Jun 7 14:08:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:32.704452 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225b550-847e-45b5-981d-2c3d0042bb4b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:32.786594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:32.813324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225b550-847e-45b5-981d-2c3d0042bb4b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:32.848048 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ca42b-61e6-44b0-852f-f4ef7333ca35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:32.848846 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ca42b-61e6-44b0-852f-f4ef7333ca35 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:32.849132 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ca42b-61e6-44b0-852f-f4ef7333ca35 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:32.850088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ca42b-61e6-44b0-852f-f4ef7333ca35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:32.861916 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ca42b-61e6-44b0-852f-f4ef7333ca35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:32.863416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ca42b-61e6-44b0-852f-f4ef7333ca35 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:32.882324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-ba1ca42b-61e6-44b0-852f-f4ef7333ca35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:32.882324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:32.882324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:32.882324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:32.882324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:32.882324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:32.882324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:32.882324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:32.882324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:32.882324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:32.882324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:32.882324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:32.882324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:32.882324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:32.882324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:32.883312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:32.883312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:32.883312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:32.883312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:32.883312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:32.883312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:32.883312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:32.883312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:32.883312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:32.883312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:32.883312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:32.883312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:32.883312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:32.883312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:32.883312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:32.883312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:32.883312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:32.884824 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:32.884824 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:32.884824 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:32.884824 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:32.884824 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 413/837] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:32.889317 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d6a15-688b-4c38-9092-5fe456c0a382 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:32.892937 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33fa053-a355-4214-a3a2-099f5e9f3090 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:32.893433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33fa053-a355-4214-a3a2-099f5e9f3090 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:32.894085 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33fa053-a355-4214-a3a2-099f5e9f3090 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:32.899261 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 414/838] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:32.901448 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251aa9a-6edf-40d2-9aa8-3115b4d4092e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:32.901852 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251aa9a-6edf-40d2-9aa8-3115b4d4092e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:32.902224 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251aa9a-6edf-40d2-9aa8-3115b4d4092e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:32.919269 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3251aa9a-6edf-40d2-9aa8-3115b4d4092e None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:32.920716 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3251aa9a-6edf-40d2-9aa8-3115b4d4092e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:33.001740 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 415/839] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:08:33.026460 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251aa9a-6edf-40d2-9aa8-3115b4d4092e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:33.028729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d6a15-688b-4c38-9092-5fe456c0a382 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:33.065724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e55b8-22f8-4476-be7b-426f1fd065ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:33.066738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e55b8-22f8-4476-be7b-426f1fd065ca None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:33.067270 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e55b8-22f8-4476-be7b-426f1fd065ca None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:33.067757 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e55b8-22f8-4476-be7b-426f1fd065ca None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:33.068885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e55b8-22f8-4476-be7b-426f1fd065ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:33.069426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e55b8-22f8-4476-be7b-426f1fd065ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:33.071000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e55b8-22f8-4476-be7b-426f1fd065ca None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:33.085512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 425/840] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:33.091192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e55b8-22f8-4476-be7b-426f1fd065ca None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:33.104995 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:33.147040 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:33.166190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251aa9a-6edf-40d2-9aa8-3115b4d4092e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:33.218285 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e55b8-22f8-4476-be7b-426f1fd065ca None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:33.219087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2b5d0-0342-491a-8a0c-bce598d858ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:33.219925 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2b5d0-0342-491a-8a0c-bce598d858ad None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:33.220213 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2b5d0-0342-491a-8a0c-bce598d858ad None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:33.220492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2b5d0-0342-491a-8a0c-bce598d858ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:33.234726 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b2b5d0-0342-491a-8a0c-bce598d858ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:08:33.243492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-01b2b5d0-0342-491a-8a0c-bce598d858ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:08:33.243492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:33.243492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:33.243492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:33.243492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:33.243492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:33.243492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:33.243492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:33.243492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:33.243492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:33.243492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:33.243492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:33.243492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:08:33.243492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:08:33.243492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:08:33.243492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:08:33.244436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:08:33.244436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:08:33.244436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:08:33.244436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:08:33.244436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:33.244436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:33.244436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:33.244436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:33.244436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:33.244436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:33.244436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:33.244436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:33.244436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:33.244436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:33.244436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:33.244436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:33.245555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:33.245555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:33.245555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:33.245555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:33.245555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:08:33.245555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:08:33.245555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:08:33.245555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:08:33.245555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:08:33.245555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:08:33.245555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:08:33.245555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:33.248440 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 416/841] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:08:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:08:33.256698 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2b5d0-0342-491a-8a0c-bce598d858ad None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:33.260417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5978992-080a-4341-882c-9c3de75ef170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:33.261446 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5978992-080a-4341-882c-9c3de75ef170 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:33.261899 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5978992-080a-4341-882c-9c3de75ef170 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:33.262354 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5978992-080a-4341-882c-9c3de75ef170 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:33.270120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5978992-080a-4341-882c-9c3de75ef170 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:33.272230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5978992-080a-4341-882c-9c3de75ef170 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:33.288522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-f5978992-080a-4341-882c-9c3de75ef170 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 14:08:33.288522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:33.288522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:33.288522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:33.288522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:33.288522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:33.288522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:33.288522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:33.288522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:33.288522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:33.288522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:33.288522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:33.288522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:08:33.288522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:08:33.288522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:08:33.288522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:08:33.289528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:33.289528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:33.289528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:08:33.289528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:33.289528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:08:33.289528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:33.289528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:33.289528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:33.289528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:33.289528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:33.289528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:33.289528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:33.289528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:33.289528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:33.289528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:33.289528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:33.289528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:33.289528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:33.290504 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:33.290504 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:33.290504 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:08:33.290504 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:08:33.290504 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:08:33.290504 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:08:33.290504 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:08:33.290504 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:08:33.290504 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 14:08:33.290504 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:33.293100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 426/842] 10.222.0.81 () {56 vars in 1117 bytes} [Tue Jun 7 14:08:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:33.298546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5978992-080a-4341-882c-9c3de75ef170 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:33.380984 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2b5d0-0342-491a-8a0c-bce598d858ad None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:33.402616 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:33.408465 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:33.413581 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5978992-080a-4341-882c-9c3de75ef170 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:33.426356 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ddc39-ab4b-4d98-b6f4-2d9cb8109ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:33.427465 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ddc39-ab4b-4d98-b6f4-2d9cb8109ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:33.428451 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ddc39-ab4b-4d98-b6f4-2d9cb8109ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:33.428897 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ddc39-ab4b-4d98-b6f4-2d9cb8109ba5 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:33.429917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ddc39-ab4b-4d98-b6f4-2d9cb8109ba5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:33.430403 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ddc39-ab4b-4d98-b6f4-2d9cb8109ba5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:33.432049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ddc39-ab4b-4d98-b6f4-2d9cb8109ba5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:33.442154 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 417/843] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:08:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:33.448511 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ddc39-ab4b-4d98-b6f4-2d9cb8109ba5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:33.462102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf3460-3095-4c4a-be9c-cf466a03df65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:33.463122 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf3460-3095-4c4a-be9c-cf466a03df65 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:33.463705 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf3460-3095-4c4a-be9c-cf466a03df65 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:33.464208 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf3460-3095-4c4a-be9c-cf466a03df65 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:33.465322 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf3460-3095-4c4a-be9c-cf466a03df65 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:33.465847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf3460-3095-4c4a-be9c-cf466a03df65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:33.467509 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf3460-3095-4c4a-be9c-cf466a03df65 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:33.486979 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 427/844] 10.222.0.81 () {56 vars in 1149 bytes} [Tue Jun 7 14:08:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:33.493288 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf3460-3095-4c4a-be9c-cf466a03df65 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:33.572003 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ddc39-ab4b-4d98-b6f4-2d9cb8109ba5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:33.611502 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937d834-ee38-4994-9bbf-d444086ca9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:33.612626 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937d834-ee38-4994-9bbf-d444086ca9fb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:33.613084 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937d834-ee38-4994-9bbf-d444086ca9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:33.613210 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf3460-3095-4c4a-be9c-cf466a03df65 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:33.613688 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937d834-ee38-4994-9bbf-d444086ca9fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:33.626214 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d937d834-ee38-4994-9bbf-d444086ca9fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:33.627938 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d937d834-ee38-4994-9bbf-d444086ca9fb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:33.651960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-d937d834-ee38-4994-9bbf-d444086ca9fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:33.651960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:33.651960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:33.651960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:33.651960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:33.651960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:33.651960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:33.651960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:33.651960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:33.651960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:33.651960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:33.651960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:33.651960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:33.651960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:33.651960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:33.653083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:33.653083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:33.653083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:33.653083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:33.653083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:33.653083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:33.653083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:33.653083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:33.653083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:33.653083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:33.653083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:33.653083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:33.653083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:33.653083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:33.653083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:33.653083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:33.653083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:33.653991 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:33.653991 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:33.653991 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:33.653991 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:33.654504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 418/845] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:08:33.659468 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937d834-ee38-4994-9bbf-d444086ca9fb None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:33.665913 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f39807-4cfe-43c9-bed5-f08868043955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:33.666577 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f39807-4cfe-43c9-bed5-f08868043955 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:33.666860 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f39807-4cfe-43c9-bed5-f08868043955 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:33.667134 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f39807-4cfe-43c9-bed5-f08868043955 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:33.701485 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:33.702015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:33.747976 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f39807-4cfe-43c9-bed5-f08868043955 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:33.749702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f39807-4cfe-43c9-bed5-f08868043955 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:33.778203 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937d834-ee38-4994-9bbf-d444086ca9fb None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:33.815395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db619d39-baa3-42c7-850b-3adb6b4735e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:33.816258 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db619d39-baa3-42c7-850b-3adb6b4735e4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:33.816636 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db619d39-baa3-42c7-850b-3adb6b4735e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:33.817008 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db619d39-baa3-42c7-850b-3adb6b4735e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:33.817842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db619d39-baa3-42c7-850b-3adb6b4735e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:33.818966 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db619d39-baa3-42c7-850b-3adb6b4735e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:33.820604 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db619d39-baa3-42c7-850b-3adb6b4735e4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:33.838653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 419/846] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:33.839936 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db619d39-baa3-42c7-850b-3adb6b4735e4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:33.860711 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 428/847] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:08:33 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:08:33.867871 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f39807-4cfe-43c9-bed5-f08868043955 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:33.947468 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:33.952959 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:33.969034 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db619d39-baa3-42c7-850b-3adb6b4735e4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:34.007879 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f39807-4cfe-43c9-bed5-f08868043955 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:34.022539 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf42e9-e83b-4033-9dc7-d32aa0ed59c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:34.023958 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf42e9-e83b-4033-9dc7-d32aa0ed59c7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:34.024842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf42e9-e83b-4033-9dc7-d32aa0ed59c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:34.025480 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf42e9-e83b-4033-9dc7-d32aa0ed59c7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:34.034814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf42e9-e83b-4033-9dc7-d32aa0ed59c7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:34.037747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf42e9-e83b-4033-9dc7-d32aa0ed59c7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:34.047762 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19f1b6-c4f8-4562-8baf-c5967a5fed23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:34.048683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19f1b6-c4f8-4562-8baf-c5967a5fed23 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:34.049186 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19f1b6-c4f8-4562-8baf-c5967a5fed23 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:34.049558 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19f1b6-c4f8-4562-8baf-c5967a5fed23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:34.056364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-5fdf42e9-e83b-4033-9dc7-d32aa0ed59c7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 14:08:34.056364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:34.056364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:34.056364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:34.056364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:34.056364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:34.056364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:34.056364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:34.056364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:34.056364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:34.056364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:34.056364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:34.056364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:08:34.056364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:08:34.056364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:08:34.056364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:08:34.057550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:34.057550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:34.057550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:08:34.057550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:34.057550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:08:34.057550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:34.057550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:34.057550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:34.057550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:34.057550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:34.057550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:34.057550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:34.057550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:34.057550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:34.057550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:34.057550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:34.057550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:34.057550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:34.058741 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:34.058741 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:34.058741 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:08:34.058741 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:08:34.058741 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:08:34.058741 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:08:34.058741 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:08:34.058741 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:08:34.058741 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 14:08:34.058741 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:34.062063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 420/848] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:08:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:34.066009 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad19f1b6-c4f8-4562-8baf-c5967a5fed23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:34.067740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad19f1b6-c4f8-4562-8baf-c5967a5fed23 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:34.069005 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf42e9-e83b-4033-9dc7-d32aa0ed59c7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:34.088790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-ad19f1b6-c4f8-4562-8baf-c5967a5fed23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:34.088790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:34.088790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:34.088790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:34.088790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:34.088790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:34.088790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:34.088790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:34.088790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:34.088790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:34.088790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:34.088790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:34.088790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:34.088790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:34.088790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:34.089953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:34.089953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:34.089953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:34.089953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:34.089953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:34.089953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:34.089953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:34.089953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:34.089953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:34.089953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:34.089953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:34.089953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:34.089953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:34.089953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:34.089953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:34.089953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:34.089953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:34.091093 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:34.091093 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:34.091093 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:34.091093 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:34.093142 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 429/849] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:08:34.099812 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19f1b6-c4f8-4562-8baf-c5967a5fed23 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:34.208286 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf42e9-e83b-4033-9dc7-d32aa0ed59c7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:34.242239 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19f1b6-c4f8-4562-8baf-c5967a5fed23 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:34.246291 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7cbb04-000c-433c-bd4b-d53e6b6039c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:34.246291 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cbb04-000c-433c-bd4b-d53e6b6039c0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:34.246610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cbb04-000c-433c-bd4b-d53e6b6039c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:34.246827 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cbb04-000c-433c-bd4b-d53e6b6039c0 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:34.247685 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7cbb04-000c-433c-bd4b-d53e6b6039c0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:34.247993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7cbb04-000c-433c-bd4b-d53e6b6039c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:34.249418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7cbb04-000c-433c-bd4b-d53e6b6039c0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:34.257813 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:34.275452 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 421/850] 10.222.0.81 () {56 vars in 1135 bytes} [Tue Jun 7 14:08:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:34.277129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7cbb04-000c-433c-bd4b-d53e6b6039c0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:34.291562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:34.292667 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5002e3-45fb-4ae3-a9f8-ad717d259d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:34.293057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5002e3-45fb-4ae3-a9f8-ad717d259d2c None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:34.293375 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5002e3-45fb-4ae3-a9f8-ad717d259d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:34.295804 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5002e3-45fb-4ae3-a9f8-ad717d259d2c None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:34.296666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5002e3-45fb-4ae3-a9f8-ad717d259d2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:34.296980 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5002e3-45fb-4ae3-a9f8-ad717d259d2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:34.299110 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5002e3-45fb-4ae3-a9f8-ad717d259d2c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:34.315622 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 430/851] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:34.317141 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9665e-0842-491a-a484-80a61f8a36a5 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:34.317433 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9665e-0842-491a-a484-80a61f8a36a5 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:34.317710 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9665e-0842-491a-a484-80a61f8a36a5 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:34.319360 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 431/852] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:34.324105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9665e-0842-491a-a484-80a61f8a36a5 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:34.402270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7cbb04-000c-433c-bd4b-d53e6b6039c0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:34.433447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9665e-0842-491a-a484-80a61f8a36a5 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:34.436275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd7766-3697-4ef0-b166-ed55b28b2c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:34.436969 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd7766-3697-4ef0-b166-ed55b28b2c05 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:34.437250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd7766-3697-4ef0-b166-ed55b28b2c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:34.437548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd7766-3697-4ef0-b166-ed55b28b2c05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:34.468679 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a486d9-9d22-46d9-be94-26d8b2833877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:34.469470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a486d9-9d22-46d9-be94-26d8b2833877 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:34.469748 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a486d9-9d22-46d9-be94-26d8b2833877 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:34.470021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a486d9-9d22-46d9-be94-26d8b2833877 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:34.476652 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a486d9-9d22-46d9-be94-26d8b2833877 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:34.478207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a486d9-9d22-46d9-be94-26d8b2833877 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:34.492388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-39a486d9-9d22-46d9-be94-26d8b2833877 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:34.492388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:34.492388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:34.492388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:34.492388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:34.492388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:34.492388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:34.492388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:34.492388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:34.492388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:34.492388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:34.492388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:34.492388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:08:34.492388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:08:34.492388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:08:34.493341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:08:34.493341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:34.493341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:34.493341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:34.493341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:34.493341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:34.493341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:34.493341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:34.493341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:34.493341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:34.493341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:34.493341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:34.493341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:34.493341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:34.493341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:34.493341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:34.493341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:34.494333 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:08:34.494333 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:08:34.494333 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:08:34.494333 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:08:34.494333 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:08:34.494333 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:08:34.494333 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:34.494333 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:34.494333 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 432/853] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:34.494883 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961602e-2e87-4469-a365-c147846e3ad8 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:34.495756 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961602e-2e87-4469-a365-c147846e3ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:34.495756 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961602e-2e87-4469-a365-c147846e3ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:34.513082 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-1961602e-2e87-4469-a365-c147846e3ad8 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:34.514256 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1961602e-2e87-4469-a365-c147846e3ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:34.515783 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd7766-3697-4ef0-b166-ed55b28b2c05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:34.517532 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd7766-3697-4ef0-b166-ed55b28b2c05 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:34.528328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:34.540246 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:34.589514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 433/854] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:08:34.596152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961602e-2e87-4469-a365-c147846e3ad8 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:34.629639 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 422/855] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:08:34 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:08:34.631042 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd7766-3697-4ef0-b166-ed55b28b2c05 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:34.704131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961602e-2e87-4469-a365-c147846e3ad8 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:34.732284 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd7766-3697-4ef0-b166-ed55b28b2c05 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:34.737858 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357612a-2ece-41a5-8d0f-5566ea3fef03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:34.738514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357612a-2ece-41a5-8d0f-5566ea3fef03 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:34.738792 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357612a-2ece-41a5-8d0f-5566ea3fef03 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:34.739064 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357612a-2ece-41a5-8d0f-5566ea3fef03 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:34.739955 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357612a-2ece-41a5-8d0f-5566ea3fef03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:34.740441 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357612a-2ece-41a5-8d0f-5566ea3fef03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:34.741510 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:34.742031 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357612a-2ece-41a5-8d0f-5566ea3fef03 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:34.753782 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 434/856] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:08:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:34.754689 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357612a-2ece-41a5-8d0f-5566ea3fef03 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:34.768338 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a07f99-e4a6-4138-8035-ed812b2e421e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:34.769144 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a07f99-e4a6-4138-8035-ed812b2e421e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:34.769422 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a07f99-e4a6-4138-8035-ed812b2e421e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:34.769693 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a07f99-e4a6-4138-8035-ed812b2e421e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:34.770377 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a07f99-e4a6-4138-8035-ed812b2e421e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:34.772396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a07f99-e4a6-4138-8035-ed812b2e421e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:34.784438 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-81a07f99-e4a6-4138-8035-ed812b2e421e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 14:08:34.784438 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:34.784438 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:08:34.784438 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:08:34.784438 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:08:34.784438 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:08:34.784438 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:08:34.784438 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:34.784438 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 14:08:34.784438 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:34.784438 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:08:34.784438 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:34.784438 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:34.784438 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:34.784438 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:34.784438 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:34.784438 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:34.785551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:34.785551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:34.785551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:34.785551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:34.785551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:34.785551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:34.785551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:08:34.785551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:08:34.785551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:08:34.785551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:08:34.785551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:34.785551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:34.785551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:34.785551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:34.785551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:34.785551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:34.786643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:34.786643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:34.786643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:34.786643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:34.786643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:34.786643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:34.786643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:34.786643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:34.786643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:34.786643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:34.786643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:08:34.786643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:34.786643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 14:08:34.786643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:34.786643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 423/857] 10.222.0.81 () {56 vars in 1114 bytes} [Tue Jun 7 14:08:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:34.787874 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9d448-0766-452b-b917-218909ba8bf3 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:34.787874 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9d448-0766-452b-b917-218909ba8bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:34.788002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9d448-0766-452b-b917-218909ba8bf3 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:34.789259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 424/858] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:34.789949 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9d448-0766-452b-b917-218909ba8bf3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:34.792286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:34.869341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357612a-2ece-41a5-8d0f-5566ea3fef03 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:34.894770 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9d448-0766-452b-b917-218909ba8bf3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:34.903693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff1f9f-e6bb-43c9-b268-4b7b64a294e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:34.904522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff1f9f-e6bb-43c9-b268-4b7b64a294e5 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:34.904807 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff1f9f-e6bb-43c9-b268-4b7b64a294e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:34.905082 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff1f9f-e6bb-43c9-b268-4b7b64a294e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:34.915777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ff1f9f-e6bb-43c9-b268-4b7b64a294e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:34.917203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ff1f9f-e6bb-43c9-b268-4b7b64a294e5 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:34.928179 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e9cab-2c83-427d-86f8-5528d25ed6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:34.928833 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e9cab-2c83-427d-86f8-5528d25ed6ac None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:34.929119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e9cab-2c83-427d-86f8-5528d25ed6ac None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:34.929397 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e9cab-2c83-427d-86f8-5528d25ed6ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:34.930227 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e9cab-2c83-427d-86f8-5528d25ed6ac None admin] RBAC: Adding query filter params (role.id=a72e5464b7b746978805326c224753bc, scope.project.id=f67bad575f8b4a47bee7c314497cb2fe, user.id=d00b8bc311234e6da47b365ac8a7183a) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:34.930539 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e9cab-2c83-427d-86f8-5528d25ed6ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:34.931986 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e9cab-2c83-427d-86f8-5528d25ed6ac None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:34.933843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-57ff1f9f-e6bb-43c9-b268-4b7b64a294e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:34.933843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:34.933843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:34.933843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:34.933843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:34.933843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:34.933843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:34.933843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:34.933843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:34.933843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:34.933843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:34.933843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:34.933843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:34.933843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:34.933843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:34.934770 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:34.934770 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:34.934770 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:34.934770 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:34.934770 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:34.934770 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:34.934770 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:34.934770 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:34.934770 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:34.934770 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:34.934770 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:34.934770 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:34.934770 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:34.934770 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:34.934770 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:34.934770 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:34.934770 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:34.935717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:34.935717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:34.935717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:34.935717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:34.935717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 435/859] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:34.936004 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff1f9f-e6bb-43c9-b268-4b7b64a294e5 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:34.949603 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 425/860] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:08:34 2022] GET /identity/v3/role_assignments?user.id=d00b8bc311234e6da47b365ac8a7183a&scope.project.id=f67bad575f8b4a47bee7c314497cb2fe&role.id=a72e5464b7b746978805326c224753bc => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:34.949603 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057414e8-8fa5-4b45-9549-8d69e8986451 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:34.949603 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057414e8-8fa5-4b45-9549-8d69e8986451 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:34.949603 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057414e8-8fa5-4b45-9549-8d69e8986451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:34.963129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-057414e8-8fa5-4b45-9549-8d69e8986451 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:34.964679 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-057414e8-8fa5-4b45-9549-8d69e8986451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:35.004071 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:35.020739 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:35.063255 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 426/861] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:08:35.063255 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057414e8-8fa5-4b45-9549-8d69e8986451 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:35.063572 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff1f9f-e6bb-43c9-b268-4b7b64a294e5 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:35.099666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f09af2-810d-49e0-bc24-0c98f09e82cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:35.100734 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f09af2-810d-49e0-bc24-0c98f09e82cd None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:35.101420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f09af2-810d-49e0-bc24-0c98f09e82cd None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:35.101913 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f09af2-810d-49e0-bc24-0c98f09e82cd None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:35.102677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f09af2-810d-49e0-bc24-0c98f09e82cd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:35.102962 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f09af2-810d-49e0-bc24-0c98f09e82cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:35.104397 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f09af2-810d-49e0-bc24-0c98f09e82cd None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:35.109943 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 436/862] 10.222.0.81 () {56 vars in 1114 bytes} [Tue Jun 7 14:08:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:35.110583 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f09af2-810d-49e0-bc24-0c98f09e82cd None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:35.188685 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057414e8-8fa5-4b45-9549-8d69e8986451 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:35.231361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72728e4-f7f7-4949-ba96-a972ebbdf26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:35.232043 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72728e4-f7f7-4949-ba96-a972ebbdf26f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:35.232332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72728e4-f7f7-4949-ba96-a972ebbdf26f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:35.232604 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72728e4-f7f7-4949-ba96-a972ebbdf26f None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:35.233330 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72728e4-f7f7-4949-ba96-a972ebbdf26f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:35.233638 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72728e4-f7f7-4949-ba96-a972ebbdf26f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:35.235006 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72728e4-f7f7-4949-ba96-a972ebbdf26f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:35.236749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f09af2-810d-49e0-bc24-0c98f09e82cd None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:35.251269 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 427/863] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:35.251269 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72728e4-f7f7-4949-ba96-a972ebbdf26f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:35.278976 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516313f-8c60-4559-984b-5f959b991266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:35.279762 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516313f-8c60-4559-984b-5f959b991266 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:35.280022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516313f-8c60-4559-984b-5f959b991266 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:35.280291 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516313f-8c60-4559-984b-5f959b991266 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:35.280944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516313f-8c60-4559-984b-5f959b991266 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:35.281251 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:35.282592 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516313f-8c60-4559-984b-5f959b991266 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:35.286181 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:35.298473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-9516313f-8c60-4559-984b-5f959b991266 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 14:08:35.298473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:35.298473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:08:35.298473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:08:35.298473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:08:35.298473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:08:35.298473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:08:35.298473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:35.298473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 14:08:35.298473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:35.298473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:08:35.298473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:35.298473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:35.298473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:35.298473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:35.298473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:35.299514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:35.299514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:35.299514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:35.299514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:35.299514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:35.299514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:35.299514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:35.299514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:08:35.299514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:08:35.299514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:08:35.299514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:08:35.299514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:35.299514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:35.299514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:35.299514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:35.299514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:35.300406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:35.300406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:35.300406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:35.300406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:35.300406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:35.300406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:35.300406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:35.300406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:35.300406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:35.300406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:35.300406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:35.300406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:08:35.300406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:35.300406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 14:08:35.300406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:35.303369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 437/864] 10.222.0.81 () {56 vars in 1132 bytes} [Tue Jun 7 14:08:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:35.304665 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516313f-8c60-4559-984b-5f959b991266 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:35.359819 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72728e4-f7f7-4949-ba96-a972ebbdf26f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:35.398514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abed1a-70f7-4909-b064-0b0798c7c6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:35.399183 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abed1a-70f7-4909-b064-0b0798c7c6e1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:35.399532 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abed1a-70f7-4909-b064-0b0798c7c6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:35.399886 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abed1a-70f7-4909-b064-0b0798c7c6e1 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:35.400711 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63abed1a-70f7-4909-b064-0b0798c7c6e1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:35.401013 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63abed1a-70f7-4909-b064-0b0798c7c6e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:35.402919 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63abed1a-70f7-4909-b064-0b0798c7c6e1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:35.415253 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 428/865] 10.222.0.81 () {56 vars in 1114 bytes} [Tue Jun 7 14:08:35 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:35.415253 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abed1a-70f7-4909-b064-0b0798c7c6e1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:35.415578 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516313f-8c60-4559-984b-5f959b991266 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:35.460934 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185eeaa-7721-4394-bc36-d93265926cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:35.462802 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185eeaa-7721-4394-bc36-d93265926cde None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:35.463087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185eeaa-7721-4394-bc36-d93265926cde None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:35.463420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185eeaa-7721-4394-bc36-d93265926cde None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:35.475153 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185eeaa-7721-4394-bc36-d93265926cde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:35.477056 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185eeaa-7721-4394-bc36-d93265926cde None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:35.500113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-7185eeaa-7721-4394-bc36-d93265926cde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:35.500113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:35.500113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:35.500113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:35.500113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:35.500113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:35.500113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:35.500113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:35.500113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:35.500113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:35.500113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:35.500113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:35.500113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:08:35.500113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:08:35.500113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:08:35.501103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:08:35.501103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:35.501103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:35.501103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:35.501103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:35.501103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:35.501103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:35.501103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:35.501103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:35.501103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:35.501103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:35.501103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:35.501103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:35.501103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:35.501103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:35.501103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:35.501103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:35.502016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:08:35.502016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:08:35.502016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:08:35.502016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:08:35.502016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:08:35.502016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:08:35.502016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:35.502016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:35.502016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 438/866] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:35.512314 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185eeaa-7721-4394-bc36-d93265926cde None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:35.534313 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:35.549078 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abed1a-70f7-4909-b064-0b0798c7c6e1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:35.589007 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77b67f-994c-4f28-9ef4-4c8c40ca6350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:35.589838 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77b67f-994c-4f28-9ef4-4c8c40ca6350 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:35.590125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77b67f-994c-4f28-9ef4-4c8c40ca6350 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:35.590448 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77b67f-994c-4f28-9ef4-4c8c40ca6350 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:35.591183 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c77b67f-994c-4f28-9ef4-4c8c40ca6350 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:35.591520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c77b67f-994c-4f28-9ef4-4c8c40ca6350 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:35.592940 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c77b67f-994c-4f28-9ef4-4c8c40ca6350 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:35.598292 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:35.598718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 429/867] 10.222.0.81 () {56 vars in 1126 bytes} [Tue Jun 7 14:08:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:35.605796 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77b67f-994c-4f28-9ef4-4c8c40ca6350 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:35.654906 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185eeaa-7721-4394-bc36-d93265926cde None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:35.698484 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138de7c-4a90-44e9-947c-85727569ee6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:35.699160 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138de7c-4a90-44e9-947c-85727569ee6b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:35.699980 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138de7c-4a90-44e9-947c-85727569ee6b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:35.700264 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138de7c-4a90-44e9-947c-85727569ee6b None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:35.701345 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9138de7c-4a90-44e9-947c-85727569ee6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:35.701654 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9138de7c-4a90-44e9-947c-85727569ee6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:35.703866 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9138de7c-4a90-44e9-947c-85727569ee6b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:35.722096 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 439/868] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:08:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:35.723308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77b67f-994c-4f28-9ef4-4c8c40ca6350 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:35.732228 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138de7c-4a90-44e9-947c-85727569ee6b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:35.761612 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8acfc-2050-42fc-a5c7-3ffd1b258f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:35.762291 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8acfc-2050-42fc-a5c7-3ffd1b258f59 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:35.762576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8acfc-2050-42fc-a5c7-3ffd1b258f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:35.762854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8acfc-2050-42fc-a5c7-3ffd1b258f59 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:35.763605 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8acfc-2050-42fc-a5c7-3ffd1b258f59 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:35.764093 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8acfc-2050-42fc-a5c7-3ffd1b258f59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:35.765480 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8acfc-2050-42fc-a5c7-3ffd1b258f59 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:35.775516 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 430/869] 10.222.0.81 () {56 vars in 1126 bytes} [Tue Jun 7 14:08:35 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:35.852014 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138de7c-4a90-44e9-947c-85727569ee6b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:35.879859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:35.897858 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6f665-382c-4cda-9dc9-b00121c0931d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:35.898536 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6f665-382c-4cda-9dc9-b00121c0931d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:35.898868 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6f665-382c-4cda-9dc9-b00121c0931d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:35.899152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6f665-382c-4cda-9dc9-b00121c0931d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:35.900030 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6f665-382c-4cda-9dc9-b00121c0931d None admin] RBAC: Adding query filter params (role.id=82bbe3651b9b4f378fd150d93357b2b1, scope.project.id=f67bad575f8b4a47bee7c314497cb2fe, user.id=da7c0f99060a44eaab743ba71f159e88) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:35.900341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6f665-382c-4cda-9dc9-b00121c0931d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:35.901796 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6f665-382c-4cda-9dc9-b00121c0931d None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:35.909348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 440/870] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:08:35 2022] GET /identity/v3/role_assignments?user.id=da7c0f99060a44eaab743ba71f159e88&scope.project.id=f67bad575f8b4a47bee7c314497cb2fe&role.id=82bbe3651b9b4f378fd150d93357b2b1 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:37.311648 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b44d94-cfe4-461f-9794-34761e121d71 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:37.312653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b44d94-cfe4-461f-9794-34761e121d71 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:37.313168 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b44d94-cfe4-461f-9794-34761e121d71 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:37.314983 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 431/871] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:37.320789 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169d5d6-f40f-4672-83d7-4657e720e12c None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:37.321623 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169d5d6-f40f-4672-83d7-4657e720e12c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:37.322103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169d5d6-f40f-4672-83d7-4657e720e12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:37.339934 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-5169d5d6-f40f-4672-83d7-4657e720e12c None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:37.341426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5169d5d6-f40f-4672-83d7-4657e720e12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:37.419685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 441/872] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:08:37.442962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b44d94-cfe4-461f-9794-34761e121d71 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:37.463659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:37.564528 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b44d94-cfe4-461f-9794-34761e121d71 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:37.600654 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0efa8-e148-468b-b508-90e7ece0c79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:37.601477 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0efa8-e148-468b-b508-90e7ece0c79a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:37.601757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0efa8-e148-468b-b508-90e7ece0c79a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:37.602028 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0efa8-e148-468b-b508-90e7ece0c79a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:37.613561 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0efa8-e148-468b-b508-90e7ece0c79a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:08:37.622658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-e5b0efa8-e148-468b-b508-90e7ece0c79a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 14:08:37.622658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:37.622658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:37.622658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:37.622658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:37.622658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:37.622658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:37.622658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:37.622658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:37.622658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:37.622658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:37.622658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:37.622658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:08:37.622658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:08:37.622658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:08:37.622658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:08:37.623749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:08:37.623749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:08:37.623749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:08:37.623749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:08:37.623749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:37.623749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:37.623749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:37.623749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:37.623749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:37.623749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:37.623749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:37.623749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:37.623749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:37.623749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:37.623749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:37.623749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:37.624614 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:37.624614 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:37.624614 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:37.624614 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:37.624614 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:08:37.624614 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:08:37.624614 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:08:37.624614 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:08:37.624614 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:08:37.624614 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:08:37.624614 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 14:08:37.624614 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:37.624614 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 432/873] 10.222.0.81 () {56 vars in 1120 bytes} [Tue Jun 7 14:08:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:37.636335 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169d5d6-f40f-4672-83d7-4657e720e12c None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:37.740921 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169d5d6-f40f-4672-83d7-4657e720e12c None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:37.756694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:37.780520 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f58f1-8f39-4216-9af2-77c974528cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:37.781260 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f58f1-8f39-4216-9af2-77c974528cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:37.781540 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f58f1-8f39-4216-9af2-77c974528cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:37.781865 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f58f1-8f39-4216-9af2-77c974528cd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:37.782642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f58f1-8f39-4216-9af2-77c974528cd5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:37.782947 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f58f1-8f39-4216-9af2-77c974528cd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:37.784397 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f58f1-8f39-4216-9af2-77c974528cd5 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:37.794844 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 442/874] 10.222.0.81 () {56 vars in 1115 bytes} [Tue Jun 7 14:08:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:37.800620 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0efa8-e148-468b-b508-90e7ece0c79a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:37.884636 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:37.910344 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0efa8-e148-468b-b508-90e7ece0c79a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:37.913710 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2fd7a-dfd1-4cbb-9985-3a96e9b9b85e None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:37.914016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2fd7a-dfd1-4cbb-9985-3a96e9b9b85e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:37.914272 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2fd7a-dfd1-4cbb-9985-3a96e9b9b85e None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:37.919282 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 443/875] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:37.922070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40430ff0-efd4-4290-b559-aeb730000e2d None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:37.922465 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40430ff0-efd4-4290-b559-aeb730000e2d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:37.922841 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40430ff0-efd4-4290-b559-aeb730000e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:37.941703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-40430ff0-efd4-4290-b559-aeb730000e2d None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:37.943105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-40430ff0-efd4-4290-b559-aeb730000e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:37.961888 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d73f46-7a41-4df7-b23f-0a058b04c8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:37.963208 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d73f46-7a41-4df7-b23f-0a058b04c8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:37.963682 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d73f46-7a41-4df7-b23f-0a058b04c8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:37.964101 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d73f46-7a41-4df7-b23f-0a058b04c8d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:37.978394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d73f46-7a41-4df7-b23f-0a058b04c8d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:37.980079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d73f46-7a41-4df7-b23f-0a058b04c8d6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:38.003911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-34d73f46-7a41-4df7-b23f-0a058b04c8d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:38.003911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:38.003911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:38.003911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:38.003911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:38.003911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:38.003911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:38.003911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:38.003911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:38.003911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:38.003911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:38.003911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:38.003911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:38.003911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:38.003911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:38.005026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:38.005026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:38.005026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:38.005026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:38.005026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:38.005026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:38.005026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:38.005026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:38.005026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:38.005026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:38.005026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:38.005026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:38.005026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:38.005026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:38.005026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:38.005026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:38.005026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:38.006363 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:38.006363 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:38.006363 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:38.006363 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:38.009015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 433/876] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:38.014646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d73f46-7a41-4df7-b23f-0a058b04c8d6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:38.027904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 444/877] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:08:38.047961 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40430ff0-efd4-4290-b559-aeb730000e2d None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:38.132623 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d73f46-7a41-4df7-b23f-0a058b04c8d6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:38.167243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:38.173203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40430ff0-efd4-4290-b559-aeb730000e2d None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:38.175046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349605ca-6158-454e-a699-390b422f1274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:38.175753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349605ca-6158-454e-a699-390b422f1274 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:38.176749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349605ca-6158-454e-a699-390b422f1274 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:38.176749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349605ca-6158-454e-a699-390b422f1274 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:38.177054 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349605ca-6158-454e-a699-390b422f1274 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:38.177397 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349605ca-6158-454e-a699-390b422f1274 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:38.179006 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349605ca-6158-454e-a699-390b422f1274 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:38.182604 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:38.191693 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 434/878] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:38.191693 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09820faf-3d23-40f1-bffe-88376333b02f None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:38.191693 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09820faf-3d23-40f1-bffe-88376333b02f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:38.191693 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09820faf-3d23-40f1-bffe-88376333b02f None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:38.191693 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 435/879] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:38.199664 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09820faf-3d23-40f1-bffe-88376333b02f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:38.218640 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76abf0eb-4843-44d1-8906-f43a929afd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:38.219415 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abf0eb-4843-44d1-8906-f43a929afd45 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:38.219697 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abf0eb-4843-44d1-8906-f43a929afd45 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:38.219968 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abf0eb-4843-44d1-8906-f43a929afd45 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:38.220696 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76abf0eb-4843-44d1-8906-f43a929afd45 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:38.222182 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76abf0eb-4843-44d1-8906-f43a929afd45 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:38.236433 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 445/880] 10.222.0.81 () {60 vars in 1141 bytes} [Tue Jun 7 14:08:38 2022] POST /identity/v3/services => generated 251 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:08:38.237688 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4aecf9-18ea-4913-b9a3-89f0c718990c None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:38.237976 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4aecf9-18ea-4913-b9a3-89f0c718990c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:38.238248 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4aecf9-18ea-4913-b9a3-89f0c718990c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:38.268736 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-bd4aecf9-18ea-4913-b9a3-89f0c718990c None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:38.269882 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-bd4aecf9-18ea-4913-b9a3-89f0c718990c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:38.332048 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09820faf-3d23-40f1-bffe-88376333b02f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:38.348963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 446/881] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:38 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:08:38.351144 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cdd83e-9ed9-4e11-a2dc-67ed5c2fd65a None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:38.351526 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cdd83e-9ed9-4e11-a2dc-67ed5c2fd65a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:38.351878 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cdd83e-9ed9-4e11-a2dc-67ed5c2fd65a None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:38.353417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 447/882] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:38.358895 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06297a-941c-4a51-8fd9-97d1a4b4dba8 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:38.359185 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06297a-941c-4a51-8fd9-97d1a4b4dba8 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:38.359515 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06297a-941c-4a51-8fd9-97d1a4b4dba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:38.378480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-8c06297a-941c-4a51-8fd9-97d1a4b4dba8 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:38.380073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8c06297a-941c-4a51-8fd9-97d1a4b4dba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:38.385845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652472fe-36a3-4630-b20c-2cef4b031ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:38.386975 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652472fe-36a3-4630-b20c-2cef4b031ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:38.387562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652472fe-36a3-4630-b20c-2cef4b031ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:38.388026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652472fe-36a3-4630-b20c-2cef4b031ac2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:38.396860 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652472fe-36a3-4630-b20c-2cef4b031ac2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:38.398636 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652472fe-36a3-4630-b20c-2cef4b031ac2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:38.415364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-652472fe-36a3-4630-b20c-2cef4b031ac2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 14:08:38.415364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:38.415364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:38.415364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:38.415364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:38.415364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:38.415364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:38.415364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:38.415364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:38.415364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:38.415364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:38.415364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:38.415364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:08:38.415364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:08:38.415364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:08:38.415364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:08:38.416579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:38.416579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:38.416579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:08:38.416579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:38.416579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:08:38.416579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:38.416579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:38.416579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:38.416579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:38.416579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:38.416579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:38.416579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:38.416579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:38.416579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:38.416579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:38.416579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:38.416579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:38.416579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:38.417727 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:38.417727 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:38.417727 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:08:38.417727 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:08:38.417727 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:08:38.417727 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:08:38.417727 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:08:38.417727 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:08:38.417727 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 14:08:38.417727 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:38.424435 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 436/883] 10.222.0.81 () {56 vars in 1117 bytes} [Tue Jun 7 14:08:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:38.425587 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652472fe-36a3-4630-b20c-2cef4b031ac2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:38.460731 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 448/884] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:38 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:08:38.461602 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06297a-941c-4a51-8fd9-97d1a4b4dba8 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:38.481424 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:38.492581 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:38.543898 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652472fe-36a3-4630-b20c-2cef4b031ac2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:38.574162 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06297a-941c-4a51-8fd9-97d1a4b4dba8 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:38.581126 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff23aa8-a05c-462a-a0d9-e644309e661d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:38.582451 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff23aa8-a05c-462a-a0d9-e644309e661d None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:38.582920 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff23aa8-a05c-462a-a0d9-e644309e661d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:38.583458 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff23aa8-a05c-462a-a0d9-e644309e661d None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:38.584818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff23aa8-a05c-462a-a0d9-e644309e661d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:38.586712 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff23aa8-a05c-462a-a0d9-e644309e661d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:38.605248 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 437/885] 10.222.0.81 () {60 vars in 1141 bytes} [Tue Jun 7 14:08:38 2022] POST /identity/v3/services => generated 269 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:08:38.606780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff23aa8-a05c-462a-a0d9-e644309e661d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:38.628901 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069fa18-b867-4ca8-bbb6-7c4d3ced6085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:38.629601 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069fa18-b867-4ca8-bbb6-7c4d3ced6085 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:38.629936 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069fa18-b867-4ca8-bbb6-7c4d3ced6085 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:38.630221 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069fa18-b867-4ca8-bbb6-7c4d3ced6085 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:38.631883 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1069fa18-b867-4ca8-bbb6-7c4d3ced6085 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:38.632202 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1069fa18-b867-4ca8-bbb6-7c4d3ced6085 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:38.633800 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1069fa18-b867-4ca8-bbb6-7c4d3ced6085 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:38.655284 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 449/886] 10.222.0.81 () {56 vars in 1149 bytes} [Tue Jun 7 14:08:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:38.661845 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069fa18-b867-4ca8-bbb6-7c4d3ced6085 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:38.764515 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:38.764911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff23aa8-a05c-462a-a0d9-e644309e661d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:38.774922 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:38.796778 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069fa18-b867-4ca8-bbb6-7c4d3ced6085 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:38.819887 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7181724-ac20-438e-b883-922b86137551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:38.821549 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7181724-ac20-438e-b883-922b86137551 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:38.822019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7181724-ac20-438e-b883-922b86137551 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:38.822472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7181724-ac20-438e-b883-922b86137551 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:38.824232 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7181724-ac20-438e-b883-922b86137551 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:38.826313 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7181724-ac20-438e-b883-922b86137551 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:38.839942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c9c4d-a468-4b6f-91af-0c9a210c96b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:38.840611 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c9c4d-a468-4b6f-91af-0c9a210c96b7 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:38.840894 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c9c4d-a468-4b6f-91af-0c9a210c96b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:38.841168 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c9c4d-a468-4b6f-91af-0c9a210c96b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:38.843703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-d7181724-ac20-438e-b883-922b86137551 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:08:38.843703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:38.843703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:08:38.843703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:08:38.843703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:08:38.843703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:08:38.843703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:08:38.843703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:38.843703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:08:38.843703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:38.843703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:08:38.843703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:38.843703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:38.843703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:38.843703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:38.843703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:38.843703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:38.844691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:38.844691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:38.844691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:38.844691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:38.844691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:38.844691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:38.844691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:08:38.844691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:08:38.844691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:08:38.844691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:08:38.844691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:38.844691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:38.844691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:38.844691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:38.844691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:38.844691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:38.845576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:38.845576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:38.845576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:38.845576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:38.845576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:38.845576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:38.845576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:38.845576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:38.845576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:38.845576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:38.845576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:08:38.845576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:38.845576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:08:38.845576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:38.849531 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 438/887] 10.222.0.81 () {56 vars in 1126 bytes} [Tue Jun 7 14:08:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:38.855320 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7181724-ac20-438e-b883-922b86137551 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:38.944291 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c9c4d-a468-4b6f-91af-0c9a210c96b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:38.946163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c9c4d-a468-4b6f-91af-0c9a210c96b7 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:38.996175 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7181724-ac20-438e-b883-922b86137551 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:39.034765 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4483df2-ef73-42fe-971f-c63a0fe6d5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:39.035490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4483df2-ef73-42fe-971f-c63a0fe6d5a4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:39.035776 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4483df2-ef73-42fe-971f-c63a0fe6d5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:39.036054 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4483df2-ef73-42fe-971f-c63a0fe6d5a4 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:39.036801 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4483df2-ef73-42fe-971f-c63a0fe6d5a4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:39.037112 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4483df2-ef73-42fe-971f-c63a0fe6d5a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:39.038590 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4483df2-ef73-42fe-971f-c63a0fe6d5a4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:39.040590 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:39.049006 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 439/888] 10.222.0.81 () {56 vars in 1122 bytes} [Tue Jun 7 14:08:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:39.049929 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4483df2-ef73-42fe-971f-c63a0fe6d5a4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:39.070190 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 450/889] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:08:38 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:08:39.075708 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c9c4d-a468-4b6f-91af-0c9a210c96b7 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:39.100613 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:39.196808 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4483df2-ef73-42fe-971f-c63a0fe6d5a4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:39.209816 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c9c4d-a468-4b6f-91af-0c9a210c96b7 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:39.233736 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2b612-eb2b-46f9-aab6-1bdf81af031d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:39.234784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2b612-eb2b-46f9-aab6-1bdf81af031d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:39.235300 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2b612-eb2b-46f9-aab6-1bdf81af031d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:39.235778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2b612-eb2b-46f9-aab6-1bdf81af031d None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:39.236702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2b612-eb2b-46f9-aab6-1bdf81af031d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:39.237233 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2b612-eb2b-46f9-aab6-1bdf81af031d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:39.238818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2b612-eb2b-46f9-aab6-1bdf81af031d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:39.251381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 440/890] 10.222.0.81 () {56 vars in 1122 bytes} [Tue Jun 7 14:08:39 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:39.254993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82feaed4-d66f-4207-98de-900a8d61b69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:39.256002 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82feaed4-d66f-4207-98de-900a8d61b69f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:39.256366 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82feaed4-d66f-4207-98de-900a8d61b69f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:39.256646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82feaed4-d66f-4207-98de-900a8d61b69f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:39.270567 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82feaed4-d66f-4207-98de-900a8d61b69f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:39.272268 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82feaed4-d66f-4207-98de-900a8d61b69f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:39.293549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-82feaed4-d66f-4207-98de-900a8d61b69f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:39.293549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:39.293549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:39.293549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:39.293549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:39.293549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:39.293549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:39.293549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:39.293549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:39.293549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:39.293549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:39.293549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:39.293549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:39.293549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:39.293549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:39.294533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:39.294533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:39.294533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:39.294533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:39.294533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:39.294533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:39.294533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:39.294533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:39.294533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:39.294533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:39.294533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:39.294533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:39.294533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:39.294533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:39.294533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:39.294533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:39.294533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:39.295516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:39.295516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:39.295516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:39.295516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:39.296194 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 451/891] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:08:39.302099 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2b612-eb2b-46f9-aab6-1bdf81af031d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:39.419361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:39.424932 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2b612-eb2b-46f9-aab6-1bdf81af031d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:39.464328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ed2cf-c9cc-4389-be57-7cffc9b3bfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:39.465362 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ed2cf-c9cc-4389-be57-7cffc9b3bfb1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:39.465849 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ed2cf-c9cc-4389-be57-7cffc9b3bfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:39.466337 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ed2cf-c9cc-4389-be57-7cffc9b3bfb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:39.467293 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ed2cf-c9cc-4389-be57-7cffc9b3bfb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:39.467881 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ed2cf-c9cc-4389-be57-7cffc9b3bfb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:39.469516 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ed2cf-c9cc-4389-be57-7cffc9b3bfb1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:39.484513 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 441/892] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:39.490650 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82feaed4-d66f-4207-98de-900a8d61b69f None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:39.526364 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:39.600783 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82feaed4-d66f-4207-98de-900a8d61b69f None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:39.635154 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf581c3-ac9e-46a1-a9a3-2587d53f0763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:39.636043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf581c3-ac9e-46a1-a9a3-2587d53f0763 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:39.636328 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf581c3-ac9e-46a1-a9a3-2587d53f0763 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:39.636606 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf581c3-ac9e-46a1-a9a3-2587d53f0763 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:39.645527 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf581c3-ac9e-46a1-a9a3-2587d53f0763 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:39.647792 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf581c3-ac9e-46a1-a9a3-2587d53f0763 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:39.662685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-dbf581c3-ac9e-46a1-a9a3-2587d53f0763 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:39.662685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:39.662685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:39.662685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:39.662685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:39.662685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:39.662685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:39.662685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:39.662685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:39.662685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:39.662685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:39.662685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:39.662685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:08:39.662685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:08:39.662685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:08:39.663666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:08:39.663666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:39.663666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:39.663666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:39.663666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:39.663666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:39.663666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:39.663666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:39.663666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:39.663666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:39.663666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:39.663666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:39.663666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:39.663666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:39.663666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:39.663666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:39.663666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:39.664590 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:08:39.664590 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:08:39.664590 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:08:39.664590 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:08:39.664590 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:08:39.664590 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:08:39.664590 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:39.664590 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:39.667149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 452/893] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:39.672592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ed2cf-c9cc-4389-be57-7cffc9b3bfb1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:39.777243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ed2cf-c9cc-4389-be57-7cffc9b3bfb1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:39.814219 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b7fed-3530-45d8-a587-44cd71836d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:39.814892 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b7fed-3530-45d8-a587-44cd71836d22 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:39.815175 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b7fed-3530-45d8-a587-44cd71836d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:39.815472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b7fed-3530-45d8-a587-44cd71836d22 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:39.816276 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b7fed-3530-45d8-a587-44cd71836d22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:39.816583 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b7fed-3530-45d8-a587-44cd71836d22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:39.818391 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b7fed-3530-45d8-a587-44cd71836d22 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:39.832379 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 442/894] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:08:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:39.838985 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf581c3-ac9e-46a1-a9a3-2587d53f0763 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:39.941751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:39.953025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf581c3-ac9e-46a1-a9a3-2587d53f0763 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:39.989071 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cbba2-9e1e-4e17-a3e7-b1db0828ba11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:39.989784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cbba2-9e1e-4e17-a3e7-b1db0828ba11 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:39.990072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cbba2-9e1e-4e17-a3e7-b1db0828ba11 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:39.990367 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cbba2-9e1e-4e17-a3e7-b1db0828ba11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:39.991178 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238cbba2-9e1e-4e17-a3e7-b1db0828ba11 None admin] RBAC: Adding query filter params (role.id=855dcf6914224d5c907aa41ec8c2e0c4, scope.project.id=f67bad575f8b4a47bee7c314497cb2fe, user.id=d00b8bc311234e6da47b365ac8a7183a) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:39.991517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238cbba2-9e1e-4e17-a3e7-b1db0828ba11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:39.992956 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238cbba2-9e1e-4e17-a3e7-b1db0828ba11 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:39.999613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 453/895] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:08:39 2022] GET /identity/v3/role_assignments?user.id=d00b8bc311234e6da47b365ac8a7183a&scope.project.id=f67bad575f8b4a47bee7c314497cb2fe&role.id=855dcf6914224d5c907aa41ec8c2e0c4 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:40.589102 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241c12f-e322-4d7d-aeeb-d17d41380c74 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:40.589659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241c12f-e322-4d7d-aeeb-d17d41380c74 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:40.589659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241c12f-e322-4d7d-aeeb-d17d41380c74 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:40.591170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 443/896] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:40.596740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e1d43-aa10-40dc-92dc-57ab2bf4f93b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:40.597187 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e1d43-aa10-40dc-92dc-57ab2bf4f93b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:40.597356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e1d43-aa10-40dc-92dc-57ab2bf4f93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:40.615261 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-523e1d43-aa10-40dc-92dc-57ab2bf4f93b None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:40.616802 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-523e1d43-aa10-40dc-92dc-57ab2bf4f93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:40.696594 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 454/897] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:08:40.719734 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241c12f-e322-4d7d-aeeb-d17d41380c74 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:40.741531 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:40.828137 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241c12f-e322-4d7d-aeeb-d17d41380c74 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:40.868236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc272d9-ce25-4423-ab06-7e1bb134bacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:40.868966 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc272d9-ce25-4423-ab06-7e1bb134bacb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:40.869253 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc272d9-ce25-4423-ab06-7e1bb134bacb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:40.869530 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc272d9-ce25-4423-ab06-7e1bb134bacb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:40.870441 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc272d9-ce25-4423-ab06-7e1bb134bacb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:40.871843 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc272d9-ce25-4423-ab06-7e1bb134bacb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:40.883903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-ebc272d9-ce25-4423-ab06-7e1bb134bacb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 14:08:40.883903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:40.883903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:08:40.883903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:08:40.883903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:08:40.883903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:08:40.883903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:08:40.883903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:40.883903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 14:08:40.883903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:40.883903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:08:40.883903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:40.883903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:40.883903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:40.883903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:40.883903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:40.883903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:40.884884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:40.884884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:40.884884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:40.884884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:40.884884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:40.884884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:40.884884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:08:40.884884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:08:40.884884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:08:40.884884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:08:40.884884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:40.884884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:40.884884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:40.884884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:40.884884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:40.884884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:40.885719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:40.885719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:40.885719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:40.885719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:40.885719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:40.885719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:40.885719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:40.885719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:40.885719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:40.885719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:40.885719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:08:40.885719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:40.885719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 14:08:40.885719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:40.885719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 444/898] 10.222.0.81 () {56 vars in 1114 bytes} [Tue Jun 7 14:08:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:40.893347 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e1d43-aa10-40dc-92dc-57ab2bf4f93b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:40.983828 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:40.999951 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e1d43-aa10-40dc-92dc-57ab2bf4f93b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:41.034928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeddd2a-3aa2-46dd-8973-cae73ab1c8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:41.035623 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeddd2a-3aa2-46dd-8973-cae73ab1c8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:41.035905 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeddd2a-3aa2-46dd-8973-cae73ab1c8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:41.036175 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeddd2a-3aa2-46dd-8973-cae73ab1c8a5 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:41.037549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeddd2a-3aa2-46dd-8973-cae73ab1c8a5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:41.037890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeddd2a-3aa2-46dd-8973-cae73ab1c8a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:41.039311 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeddd2a-3aa2-46dd-8973-cae73ab1c8a5 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:41.044917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 455/899] 10.222.0.81 () {56 vars in 1114 bytes} [Tue Jun 7 14:08:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:41.045587 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0482fe5-5986-49f4-94c4-d6c544b3d33b None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:41.045876 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0482fe5-5986-49f4-94c4-d6c544b3d33b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:41.046158 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0482fe5-5986-49f4-94c4-d6c544b3d33b None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:41.051148 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 445/900] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:41.052219 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeddd2a-3aa2-46dd-8973-cae73ab1c8a5 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:41.053870 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43bff6-b3f1-422b-8cfe-fe37a9139f19 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:41.054157 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43bff6-b3f1-422b-8cfe-fe37a9139f19 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:41.054433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43bff6-b3f1-422b-8cfe-fe37a9139f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:41.076239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-cb43bff6-b3f1-422b-8cfe-fe37a9139f19 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:41.077866 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cb43bff6-b3f1-422b-8cfe-fe37a9139f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:41.177236 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeddd2a-3aa2-46dd-8973-cae73ab1c8a5 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:41.177700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 446/901] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:08:41.197059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43bff6-b3f1-422b-8cfe-fe37a9139f19 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:41.213274 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:41.215104 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef363c62-e627-4454-9ad8-9c25b5d28e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:41.215836 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef363c62-e627-4454-9ad8-9c25b5d28e0b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:41.216215 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef363c62-e627-4454-9ad8-9c25b5d28e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:41.216593 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef363c62-e627-4454-9ad8-9c25b5d28e0b None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:41.217376 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef363c62-e627-4454-9ad8-9c25b5d28e0b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:41.217683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef363c62-e627-4454-9ad8-9c25b5d28e0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:41.219233 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef363c62-e627-4454-9ad8-9c25b5d28e0b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:41.225188 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 456/902] 10.222.0.81 () {56 vars in 1114 bytes} [Tue Jun 7 14:08:41 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:41.231384 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef363c62-e627-4454-9ad8-9c25b5d28e0b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:41.251970 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:41.314051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43bff6-b3f1-422b-8cfe-fe37a9139f19 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:41.353582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95e0b1-4cfb-47f3-8a41-b1a0f4f385cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:41.354932 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95e0b1-4cfb-47f3-8a41-b1a0f4f385cd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:41.355428 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95e0b1-4cfb-47f3-8a41-b1a0f4f385cd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:41.355898 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95e0b1-4cfb-47f3-8a41-b1a0f4f385cd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:41.356728 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95e0b1-4cfb-47f3-8a41-b1a0f4f385cd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:41.358495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95e0b1-4cfb-47f3-8a41-b1a0f4f385cd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:41.365015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef363c62-e627-4454-9ad8-9c25b5d28e0b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:41.373199 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-9e95e0b1-4cfb-47f3-8a41-b1a0f4f385cd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 14:08:41.373199 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:41.373199 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:08:41.373199 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:08:41.373199 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:08:41.373199 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:08:41.373199 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:08:41.373199 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:41.373199 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 14:08:41.373199 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:41.373199 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:08:41.373199 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:41.373199 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:41.373199 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:41.373199 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:41.373199 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:41.374240 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:41.374240 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:41.374240 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:41.374240 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:41.374240 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:41.374240 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:41.374240 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:41.374240 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:08:41.374240 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:08:41.374240 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:08:41.374240 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:08:41.374240 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:41.374240 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:41.374240 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:41.374240 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:41.374240 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:41.375094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:41.375094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:41.375094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:41.375094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:41.375094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:41.375094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:41.375094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:41.375094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:41.375094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:41.375094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:41.375094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:41.375094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:08:41.375094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:41.375094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 14:08:41.375094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:41.383269 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 447/903] 10.222.0.81 () {56 vars in 1132 bytes} [Tue Jun 7 14:08:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:41.384552 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95e0b1-4cfb-47f3-8a41-b1a0f4f385cd None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:41.413315 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024b2f5-239a-4dda-91bb-55a0f184e6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:41.414514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024b2f5-239a-4dda-91bb-55a0f184e6d2 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:41.414893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024b2f5-239a-4dda-91bb-55a0f184e6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:41.415254 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024b2f5-239a-4dda-91bb-55a0f184e6d2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:41.416409 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024b2f5-239a-4dda-91bb-55a0f184e6d2 None admin] RBAC: Adding query filter params (interface=public, service_id=f52d391366ba4a8b83730d03fe7d3ef9, region_id=RegionOne) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:41.416793 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024b2f5-239a-4dda-91bb-55a0f184e6d2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:41.418997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024b2f5-239a-4dda-91bb-55a0f184e6d2 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:41.427076 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 457/904] 10.222.0.81 () {56 vars in 1251 bytes} [Tue Jun 7 14:08:41 2022] GET /identity/v3/endpoints?service_id=f52d391366ba4a8b83730d03fe7d3ef9&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:41.431115 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024b2f5-239a-4dda-91bb-55a0f184e6d2 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:41.479400 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:41.506060 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95e0b1-4cfb-47f3-8a41-b1a0f4f385cd None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:41.543701 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f5eb8-226b-4044-bd5b-587370df24b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:41.544671 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f5eb8-226b-4044-bd5b-587370df24b3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:41.545156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f5eb8-226b-4044-bd5b-587370df24b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:41.545431 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:41.546214 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f5eb8-226b-4044-bd5b-587370df24b3 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:41.547252 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f5eb8-226b-4044-bd5b-587370df24b3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:41.548505 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f5eb8-226b-4044-bd5b-587370df24b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:41.549370 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f5eb8-226b-4044-bd5b-587370df24b3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:41.561130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 448/905] 10.222.0.81 () {56 vars in 1126 bytes} [Tue Jun 7 14:08:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:41.562472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f5eb8-226b-4044-bd5b-587370df24b3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:41.575687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024b2f5-239a-4dda-91bb-55a0f184e6d2 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:41.623705 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0bb64-f908-4715-8f1f-3728da279ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:41.624362 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0bb64-f908-4715-8f1f-3728da279ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:41.624723 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0bb64-f908-4715-8f1f-3728da279ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:41.625003 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0bb64-f908-4715-8f1f-3728da279ab9 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:41.626146 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0bb64-f908-4715-8f1f-3728da279ab9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:41.628254 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0bb64-f908-4715-8f1f-3728da279ab9 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:41.637594 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 458/906] 10.222.0.81 () {56 vars in 1087 bytes} [Tue Jun 7 14:08:41 2022] GET /identity/v3/services => generated 1268 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:08:41.682951 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f5eb8-226b-4044-bd5b-587370df24b3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:41.723206 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77789a5-9757-4505-b5aa-6ecfe66cbd09 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:41.723680 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77789a5-9757-4505-b5aa-6ecfe66cbd09 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:41.724614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77789a5-9757-4505-b5aa-6ecfe66cbd09 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:41.725655 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef403be1-7457-4d7b-b7b8-5cabbd919197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:41.726501 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef403be1-7457-4d7b-b7b8-5cabbd919197 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:41.726930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef403be1-7457-4d7b-b7b8-5cabbd919197 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:41.727390 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef403be1-7457-4d7b-b7b8-5cabbd919197 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:41.728262 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef403be1-7457-4d7b-b7b8-5cabbd919197 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:41.728699 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef403be1-7457-4d7b-b7b8-5cabbd919197 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:41.730238 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef403be1-7457-4d7b-b7b8-5cabbd919197 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:41.731153 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 459/907] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:08:41.732974 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac436e9-c687-42f3-b0e4-fd6663c8b7e1 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:41.733656 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac436e9-c687-42f3-b0e4-fd6663c8b7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:41.733656 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac436e9-c687-42f3-b0e4-fd6663c8b7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:41.737419 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 449/908] 10.222.0.81 () {56 vars in 1126 bytes} [Tue Jun 7 14:08:41 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:41.744200 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef403be1-7457-4d7b-b7b8-5cabbd919197 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:41.748898 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:41.753838 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-2ac436e9-c687-42f3-b0e4-fd6663c8b7e1 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:41.755310 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2ac436e9-c687-42f3-b0e4-fd6663c8b7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:41.839337 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 460/909] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:08:41.857666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac436e9-c687-42f3-b0e4-fd6663c8b7e1 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:41.865872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef403be1-7457-4d7b-b7b8-5cabbd919197 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:41.910377 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5f5ce-8061-45f2-b7d9-6d852eb93d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:41.912414 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5f5ce-8061-45f2-b7d9-6d852eb93d6f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:41.913257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5f5ce-8061-45f2-b7d9-6d852eb93d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:41.913736 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5f5ce-8061-45f2-b7d9-6d852eb93d6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:41.915004 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d5f5ce-8061-45f2-b7d9-6d852eb93d6f None admin] RBAC: Adding query filter params (interface=public, service_id=93c1ff73c85641d58ec874955ce003ec, region_id=RegionOne) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:41.915922 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d5f5ce-8061-45f2-b7d9-6d852eb93d6f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:41.917767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d5f5ce-8061-45f2-b7d9-6d852eb93d6f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:41.928973 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 450/910] 10.222.0.81 () {56 vars in 1251 bytes} [Tue Jun 7 14:08:41 2022] GET /identity/v3/endpoints?service_id=93c1ff73c85641d58ec874955ce003ec&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:41.934725 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5f5ce-8061-45f2-b7d9-6d852eb93d6f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:41.944415 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:41.975898 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac436e9-c687-42f3-b0e4-fd6663c8b7e1 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:42.011584 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:42.015984 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585488b-e8c7-4ef6-8148-b754d45fa5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:42.016794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585488b-e8c7-4ef6-8148-b754d45fa5de None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:42.017930 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585488b-e8c7-4ef6-8148-b754d45fa5de None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:42.017930 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585488b-e8c7-4ef6-8148-b754d45fa5de None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:42.018349 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585488b-e8c7-4ef6-8148-b754d45fa5de None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:42.019882 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585488b-e8c7-4ef6-8148-b754d45fa5de None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:42.037167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 461/911] 10.222.0.81 () {60 vars in 1141 bytes} [Tue Jun 7 14:08:41 2022] POST /identity/v3/services => generated 241 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:08:42.051196 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5f5ce-8061-45f2-b7d9-6d852eb93d6f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:42.103555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31f26d-4eae-449f-a042-edb1c0b64df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:42.104305 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31f26d-4eae-449f-a042-edb1c0b64df3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:42.104644 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31f26d-4eae-449f-a042-edb1c0b64df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:42.104927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31f26d-4eae-449f-a042-edb1c0b64df3 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:42.105628 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af31f26d-4eae-449f-a042-edb1c0b64df3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:42.107299 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af31f26d-4eae-449f-a042-edb1c0b64df3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:42.115515 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 451/912] 10.222.0.81 () {56 vars in 1087 bytes} [Tue Jun 7 14:08:41 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:08:42.344279 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b064f96-9147-4a11-a400-bc017f6bc988 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:42.344572 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b064f96-9147-4a11-a400-bc017f6bc988 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:42.344854 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b064f96-9147-4a11-a400-bc017f6bc988 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:42.346437 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 462/913] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:42.352628 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc4940-a01b-425b-a213-cbb838f2db31 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:42.352916 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc4940-a01b-425b-a213-cbb838f2db31 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:42.353193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc4940-a01b-425b-a213-cbb838f2db31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:42.371952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4fcc4940-a01b-425b-a213-cbb838f2db31 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:42.373137 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4fcc4940-a01b-425b-a213-cbb838f2db31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:42.446918 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 452/914] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:08:42.469705 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b064f96-9147-4a11-a400-bc017f6bc988 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:42.587204 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b064f96-9147-4a11-a400-bc017f6bc988 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:42.623872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:42.626919 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d88b7-ab3b-4710-831d-4004cbe529c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:42.627895 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d88b7-ab3b-4710-831d-4004cbe529c6 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:42.628351 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d88b7-ab3b-4710-831d-4004cbe529c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:42.629021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d88b7-ab3b-4710-831d-4004cbe529c6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:42.640908 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964d88b7-ab3b-4710-831d-4004cbe529c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:08:42.650037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-964d88b7-ab3b-4710-831d-4004cbe529c6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 14:08:42.650037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:42.650037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:42.650037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:42.650037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:42.650037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:42.650037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:42.650037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:42.650037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:42.650037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:42.650037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:42.650037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:42.650037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:08:42.650037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:08:42.650037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:08:42.650037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:08:42.650981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:08:42.650981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:08:42.650981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:08:42.650981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:08:42.650981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:42.650981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:42.650981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:42.650981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:42.650981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:42.650981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:42.650981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:42.650981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:42.650981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:42.650981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:42.650981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:42.650981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:42.651893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:42.651893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:42.651893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:42.651893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:42.651893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:08:42.651893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:08:42.651893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:08:42.651893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:08:42.651893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:08:42.651893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:08:42.651893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 14:08:42.651893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:42.651893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 463/915] 10.222.0.81 () {56 vars in 1120 bytes} [Tue Jun 7 14:08:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:42.658958 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc4940-a01b-425b-a213-cbb838f2db31 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:42.725232 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:42.775112 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc4940-a01b-425b-a213-cbb838f2db31 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:42.812657 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01177c41-8a0e-4b7a-bbbf-07d5f4a4854e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:42.813338 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01177c41-8a0e-4b7a-bbbf-07d5f4a4854e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:42.813621 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01177c41-8a0e-4b7a-bbbf-07d5f4a4854e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:42.813897 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01177c41-8a0e-4b7a-bbbf-07d5f4a4854e None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:42.814656 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01177c41-8a0e-4b7a-bbbf-07d5f4a4854e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:42.814963 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01177c41-8a0e-4b7a-bbbf-07d5f4a4854e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:42.816380 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01177c41-8a0e-4b7a-bbbf-07d5f4a4854e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:42.832686 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 453/916] 10.222.0.81 () {56 vars in 1115 bytes} [Tue Jun 7 14:08:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:42.840320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d88b7-ab3b-4710-831d-4004cbe529c6 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:42.952347 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d88b7-ab3b-4710-831d-4004cbe529c6 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:42.988397 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7e16e-6886-4bfa-962b-ffa7a2c7199a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:42.989196 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7e16e-6886-4bfa-962b-ffa7a2c7199a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:42.989482 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7e16e-6886-4bfa-962b-ffa7a2c7199a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:42.990348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7e16e-6886-4bfa-962b-ffa7a2c7199a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:43.002368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7e16e-6886-4bfa-962b-ffa7a2c7199a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:43.003867 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7e16e-6886-4bfa-962b-ffa7a2c7199a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:43.021784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-a7f7e16e-6886-4bfa-962b-ffa7a2c7199a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:43.021784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:43.021784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:43.021784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:43.021784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:43.021784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:43.021784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:43.021784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:43.021784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:43.021784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:43.021784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:43.021784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:43.021784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:43.021784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:43.021784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:43.022760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:43.022760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:43.022760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:43.022760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:43.022760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:43.022760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:43.022760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:43.022760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:43.022760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:43.022760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:43.022760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:43.022760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:43.022760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:43.022760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:43.022760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:43.022760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:43.022760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:43.023703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:43.023703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:43.023703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:43.023703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:43.023703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 464/917] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:43.032482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01177c41-8a0e-4b7a-bbbf-07d5f4a4854e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:43.147522 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01177c41-8a0e-4b7a-bbbf-07d5f4a4854e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:43.173152 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:43.186651 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d713824-837f-4ce6-bbd5-aa17035e06a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:43.187347 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d713824-837f-4ce6-bbd5-aa17035e06a7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:43.187638 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d713824-837f-4ce6-bbd5-aa17035e06a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:43.187916 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d713824-837f-4ce6-bbd5-aa17035e06a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:43.188645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d713824-837f-4ce6-bbd5-aa17035e06a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:43.188949 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d713824-837f-4ce6-bbd5-aa17035e06a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:43.190333 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d713824-837f-4ce6-bbd5-aa17035e06a7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:43.203157 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 454/918] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:43.208660 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7e16e-6886-4bfa-962b-ffa7a2c7199a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:43.261215 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:43.319176 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7e16e-6886-4bfa-962b-ffa7a2c7199a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:43.355007 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34227f58-b4be-48e2-bdbb-8b1d6ec00aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:43.355854 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34227f58-b4be-48e2-bdbb-8b1d6ec00aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:43.356146 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34227f58-b4be-48e2-bdbb-8b1d6ec00aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:43.356425 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34227f58-b4be-48e2-bdbb-8b1d6ec00aa5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:43.364550 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34227f58-b4be-48e2-bdbb-8b1d6ec00aa5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:43.366690 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34227f58-b4be-48e2-bdbb-8b1d6ec00aa5 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:43.382857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-34227f58-b4be-48e2-bdbb-8b1d6ec00aa5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 14:08:43.382857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:43.382857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:43.382857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:43.382857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:43.382857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:43.382857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:43.382857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:43.382857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:43.382857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:43.382857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:43.382857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:43.382857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:08:43.382857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:08:43.382857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:08:43.382857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:08:43.383873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:43.383873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:43.383873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:08:43.383873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:43.383873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:08:43.383873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:43.383873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:43.383873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:43.383873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:43.383873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:43.383873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:43.383873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:43.383873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:43.383873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:43.383873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:43.383873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:43.383873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:43.383873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:43.384861 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:43.384861 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:43.384861 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:08:43.384861 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:08:43.384861 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:08:43.384861 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:08:43.384861 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:08:43.384861 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:08:43.384861 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 14:08:43.384861 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:43.384861 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 465/919] 10.222.0.81 () {56 vars in 1117 bytes} [Tue Jun 7 14:08:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:43.392338 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d713824-837f-4ce6-bbd5-aa17035e06a7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:43.508598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d713824-837f-4ce6-bbd5-aa17035e06a7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:43.544252 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaa55db-4dd0-41cf-9600-b4553a21566a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:43.544935 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa55db-4dd0-41cf-9600-b4553a21566a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:43.545221 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa55db-4dd0-41cf-9600-b4553a21566a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:43.545495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa55db-4dd0-41cf-9600-b4553a21566a None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:43.546335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaa55db-4dd0-41cf-9600-b4553a21566a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:43.546641 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaa55db-4dd0-41cf-9600-b4553a21566a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:43.548110 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaa55db-4dd0-41cf-9600-b4553a21566a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:43.567140 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 455/920] 10.222.0.81 () {56 vars in 1149 bytes} [Tue Jun 7 14:08:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:43.576318 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34227f58-b4be-48e2-bdbb-8b1d6ec00aa5 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:43.685727 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34227f58-b4be-48e2-bdbb-8b1d6ec00aa5 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:43.694049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:43.725892 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea41393-68b1-43c4-a3df-918f0593c250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:43.726709 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea41393-68b1-43c4-a3df-918f0593c250 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:43.726991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea41393-68b1-43c4-a3df-918f0593c250 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:43.727295 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea41393-68b1-43c4-a3df-918f0593c250 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:43.738683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea41393-68b1-43c4-a3df-918f0593c250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:43.740763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea41393-68b1-43c4-a3df-918f0593c250 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:43.760634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-3ea41393-68b1-43c4-a3df-918f0593c250 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:43.760634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:43.760634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:43.760634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:43.760634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:43.760634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:43.760634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:43.760634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:43.760634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:43.760634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:43.760634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:43.760634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:43.760634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:43.760634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:43.760634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:43.761565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:43.761565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:43.761565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:43.761565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:43.761565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:43.761565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:43.761565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:43.761565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:43.761565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:43.761565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:43.761565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:43.761565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:43.761565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:43.761565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:43.761565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:43.761565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:43.761565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:43.762476 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:43.762476 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:43.762476 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:43.762476 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:43.762476 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 466/921] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:43.768966 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaa55db-4dd0-41cf-9600-b4553a21566a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:43.811731 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:43.874664 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaa55db-4dd0-41cf-9600-b4553a21566a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:43.913018 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e77dad-6310-4e9b-8ff4-97035e117c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:43.913697 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e77dad-6310-4e9b-8ff4-97035e117c5c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:43.913980 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e77dad-6310-4e9b-8ff4-97035e117c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:43.914258 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e77dad-6310-4e9b-8ff4-97035e117c5c None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:43.914993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e77dad-6310-4e9b-8ff4-97035e117c5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:43.915381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e77dad-6310-4e9b-8ff4-97035e117c5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:43.916758 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e77dad-6310-4e9b-8ff4-97035e117c5c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:43.930118 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 456/922] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:43.936332 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea41393-68b1-43c4-a3df-918f0593c250 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:44.044788 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea41393-68b1-43c4-a3df-918f0593c250 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:44.056755 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c656f-4dba-486f-b6ba-26bbdeb2343e None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:44.057098 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c656f-4dba-486f-b6ba-26bbdeb2343e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:44.057331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c656f-4dba-486f-b6ba-26bbdeb2343e None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:44.058734 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 457/923] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:44.066787 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a35d77-7ad5-48d7-ad6e-eff2717d657b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:44.067088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a35d77-7ad5-48d7-ad6e-eff2717d657b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:44.067424 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a35d77-7ad5-48d7-ad6e-eff2717d657b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:44.086292 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e0a35d77-7ad5-48d7-ad6e-eff2717d657b None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:44.087561 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e0a35d77-7ad5-48d7-ad6e-eff2717d657b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:44.089350 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9198105-3917-47b1-94e4-19c3bfe8aa7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:44.090221 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9198105-3917-47b1-94e4-19c3bfe8aa7d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:44.090544 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9198105-3917-47b1-94e4-19c3bfe8aa7d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:44.090822 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9198105-3917-47b1-94e4-19c3bfe8aa7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:44.098389 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9198105-3917-47b1-94e4-19c3bfe8aa7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:44.100851 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9198105-3917-47b1-94e4-19c3bfe8aa7d None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:44.115751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-b9198105-3917-47b1-94e4-19c3bfe8aa7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:44.115751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:44.115751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:44.115751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:44.115751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:44.115751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:44.115751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:44.115751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:44.115751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:44.115751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:44.115751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:44.115751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:44.115751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:08:44.115751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:08:44.115751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:08:44.116707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:08:44.116707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:44.116707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:44.116707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:44.116707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:44.116707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:44.116707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:44.116707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:44.116707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:44.116707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:44.116707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:44.116707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:44.116707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:44.116707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:44.116707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:44.116707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:44.116707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:44.117622 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:08:44.117622 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:08:44.117622 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:08:44.117622 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:08:44.117622 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:08:44.117622 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:08:44.117622 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:44.117622 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:44.117622 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 467/924] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:44.125065 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9198105-3917-47b1-94e4-19c3bfe8aa7d None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:44.160999 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:44.172222 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 458/925] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:08:44.195587 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a35d77-7ad5-48d7-ad6e-eff2717d657b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:44.232624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:44.262049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9198105-3917-47b1-94e4-19c3bfe8aa7d None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:44.303774 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9b057-d794-451c-b03d-071cecbcad41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:44.304380 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9b057-d794-451c-b03d-071cecbcad41 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:44.304646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9b057-d794-451c-b03d-071cecbcad41 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:44.305033 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9b057-d794-451c-b03d-071cecbcad41 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:44.305830 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9b057-d794-451c-b03d-071cecbcad41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:44.306143 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9b057-d794-451c-b03d-071cecbcad41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:44.312131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9b057-d794-451c-b03d-071cecbcad41 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:44.333041 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 468/926] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:08:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:44.334789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a35d77-7ad5-48d7-ad6e-eff2717d657b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:44.335396 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9b057-d794-451c-b03d-071cecbcad41 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:44.375753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1207f7b-aafb-4037-8dcf-3cdf02dd2edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:44.376630 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1207f7b-aafb-4037-8dcf-3cdf02dd2edb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:44.376924 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1207f7b-aafb-4037-8dcf-3cdf02dd2edb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:44.377200 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1207f7b-aafb-4037-8dcf-3cdf02dd2edb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:44.377862 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1207f7b-aafb-4037-8dcf-3cdf02dd2edb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:44.379319 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1207f7b-aafb-4037-8dcf-3cdf02dd2edb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:44.392883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-d1207f7b-aafb-4037-8dcf-3cdf02dd2edb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 14:08:44.392883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:44.392883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:08:44.392883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:08:44.392883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:08:44.392883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:08:44.392883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:08:44.392883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:44.392883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 14:08:44.392883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:44.392883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:08:44.392883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:44.392883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:44.392883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:44.392883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:44.392883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:44.392883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:44.393930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:44.393930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:44.393930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:44.393930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:44.393930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:44.393930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:44.393930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:08:44.393930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:08:44.393930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:08:44.393930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:08:44.393930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:44.393930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:44.393930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:44.393930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:44.393930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:44.393930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:44.394800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:44.394800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:44.394800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:44.394800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:44.394800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:44.394800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:44.394800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:44.394800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:44.394800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:44.394800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:44.394800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:08:44.394800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:44.394800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 14:08:44.394800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:44.399351 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 459/927] 10.222.0.81 () {56 vars in 1114 bytes} [Tue Jun 7 14:08:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:44.401362 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1207f7b-aafb-4037-8dcf-3cdf02dd2edb None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:44.441120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:44.467377 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9b057-d794-451c-b03d-071cecbcad41 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:44.499286 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:44.505550 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02709c06-b30e-45af-8ea9-02321a6811f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:44.506653 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02709c06-b30e-45af-8ea9-02321a6811f0 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:44.506921 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02709c06-b30e-45af-8ea9-02321a6811f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:44.507209 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02709c06-b30e-45af-8ea9-02321a6811f0 None admin] PATH_INFO: `/v3/projects/f67bad575f8b4a47bee7c314497cb2fe/users/d00b8bc311234e6da47b365ac8a7183a/roles/855dcf6914224d5c907aa41ec8c2e0c4` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:44.509267 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1207f7b-aafb-4037-8dcf-3cdf02dd2edb None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:44.509459 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02709c06-b30e-45af-8ea9-02321a6811f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=f67bad575f8b4a47bee7c314497cb2fe, user_id=d00b8bc311234e6da47b365ac8a7183a, role_id=855dcf6914224d5c907aa41ec8c2e0c4)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:44.510838 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02709c06-b30e-45af-8ea9-02321a6811f0 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:44.531265 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 469/928] 10.222.0.81 () {58 vars in 1439 bytes} [Tue Jun 7 14:08:44 2022] PUT /identity/v3/projects/f67bad575f8b4a47bee7c314497cb2fe/users/d00b8bc311234e6da47b365ac8a7183a/roles/855dcf6914224d5c907aa41ec8c2e0c4 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:08:44.531265 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae396010-2a64-4eb6-8d76-dae081c228a1 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:44.531265 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae396010-2a64-4eb6-8d76-dae081c228a1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:44.531265 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae396010-2a64-4eb6-8d76-dae081c228a1 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:44.531265 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 470/929] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:44.533406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3521a0b-0be2-4eb0-9c61-14cb22870bc4 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:44.533683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3521a0b-0be2-4eb0-9c61-14cb22870bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:44.534339 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3521a0b-0be2-4eb0-9c61-14cb22870bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:44.554729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-f3521a0b-0be2-4eb0-9c61-14cb22870bc4 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:44.555883 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f3521a0b-0be2-4eb0-9c61-14cb22870bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:44.557427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4381926-8793-40c7-9289-a1fc798634b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:44.558107 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4381926-8793-40c7-9289-a1fc798634b7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:44.558390 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4381926-8793-40c7-9289-a1fc798634b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:44.558664 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4381926-8793-40c7-9289-a1fc798634b7 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:44.559474 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4381926-8793-40c7-9289-a1fc798634b7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:44.559782 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4381926-8793-40c7-9289-a1fc798634b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:44.561251 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4381926-8793-40c7-9289-a1fc798634b7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:44.568778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 460/930] 10.222.0.81 () {56 vars in 1114 bytes} [Tue Jun 7 14:08:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:44.575618 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4381926-8793-40c7-9289-a1fc798634b7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:44.663502 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 471/931] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:08:44.686568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3521a0b-0be2-4eb0-9c61-14cb22870bc4 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:44.704695 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4381926-8793-40c7-9289-a1fc798634b7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:44.743857 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51739ba7-1aa5-4f24-a722-e23eb995aa52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:44.744620 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51739ba7-1aa5-4f24-a722-e23eb995aa52 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:44.745066 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51739ba7-1aa5-4f24-a722-e23eb995aa52 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:44.745348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51739ba7-1aa5-4f24-a722-e23eb995aa52 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:44.746067 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51739ba7-1aa5-4f24-a722-e23eb995aa52 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:44.746375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51739ba7-1aa5-4f24-a722-e23eb995aa52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:44.747789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51739ba7-1aa5-4f24-a722-e23eb995aa52 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:44.756053 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 461/932] 10.222.0.81 () {56 vars in 1114 bytes} [Tue Jun 7 14:08:44 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:44.756547 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7934e87-a47e-4c25-8082-472bdcd6cd62 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:44.756837 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7934e87-a47e-4c25-8082-472bdcd6cd62 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:44.757114 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7934e87-a47e-4c25-8082-472bdcd6cd62 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:44.758359 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 462/933] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:44.761123 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7934e87-a47e-4c25-8082-472bdcd6cd62 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:44.777754 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:44.788781 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:44.819105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3521a0b-0be2-4eb0-9c61-14cb22870bc4 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:44.856409 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5c389-74b0-4c59-87b8-08deaf672930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:44.857267 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5c389-74b0-4c59-87b8-08deaf672930 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:44.857548 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5c389-74b0-4c59-87b8-08deaf672930 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:44.857917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5c389-74b0-4c59-87b8-08deaf672930 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:44.858468 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5c389-74b0-4c59-87b8-08deaf672930 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:44.859879 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5c389-74b0-4c59-87b8-08deaf672930 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:44.873628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-d4f5c389-74b0-4c59-87b8-08deaf672930 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:08:44.873628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:44.873628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:08:44.873628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:08:44.873628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:08:44.873628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:08:44.873628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:08:44.873628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:44.873628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:08:44.873628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:44.873628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:08:44.873628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:44.873628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:44.873628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:44.873628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:44.873628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:44.873628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:44.874946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:44.874946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:44.874946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:44.874946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:44.874946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:44.874946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:44.874946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:08:44.874946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:08:44.874946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:08:44.874946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:08:44.874946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:44.874946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:44.874946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:44.874946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:44.874946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:44.874946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:44.876070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:44.876070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:44.876070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:44.876070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:44.876070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:44.876070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:44.876070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:44.876070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:44.876070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:44.876070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:44.876070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:08:44.876070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:44.876070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:08:44.876070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:44.877976 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 472/934] 10.222.0.81 () {56 vars in 1126 bytes} [Tue Jun 7 14:08:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:08:44.879099 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe102cdb-0e7a-4ce5-8ebf-b64a8df75592 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:44.879472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe102cdb-0e7a-4ce5-8ebf-b64a8df75592 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:44.879857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe102cdb-0e7a-4ce5-8ebf-b64a8df75592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:44.880972 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7934e87-a47e-4c25-8082-472bdcd6cd62 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:44.897359 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-fe102cdb-0e7a-4ce5-8ebf-b64a8df75592 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:44.898546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-fe102cdb-0e7a-4ce5-8ebf-b64a8df75592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:44.918491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab765e-cc81-43dd-a97d-eaa6db19e962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:44.919233 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab765e-cc81-43dd-a97d-eaa6db19e962 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:44.919539 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab765e-cc81-43dd-a97d-eaa6db19e962 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:44.920247 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab765e-cc81-43dd-a97d-eaa6db19e962 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:44.921284 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ab765e-cc81-43dd-a97d-eaa6db19e962 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:44.922830 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ab765e-cc81-43dd-a97d-eaa6db19e962 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:44.949184 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 463/935] 10.222.0.81 () {60 vars in 1144 bytes} [Tue Jun 7 14:08:44 2022] POST /identity/v3/endpoints => generated 346 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:08:44.950019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab765e-cc81-43dd-a97d-eaa6db19e962 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:45.001473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 473/936] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:08:45.035405 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe102cdb-0e7a-4ce5-8ebf-b64a8df75592 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:45.055064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:45.094946 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab765e-cc81-43dd-a97d-eaa6db19e962 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:45.108089 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:45.148524 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc3e0b0-7b96-4654-b827-d6c65f4e3be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:45.149886 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3e0b0-7b96-4654-b827-d6c65f4e3be9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:45.150860 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3e0b0-7b96-4654-b827-d6c65f4e3be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:45.151676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3e0b0-7b96-4654-b827-d6c65f4e3be9 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:45.152657 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc3e0b0-7b96-4654-b827-d6c65f4e3be9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:45.153172 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc3e0b0-7b96-4654-b827-d6c65f4e3be9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:45.154806 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc3e0b0-7b96-4654-b827-d6c65f4e3be9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:45.164736 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 464/937] 10.222.0.81 () {56 vars in 1122 bytes} [Tue Jun 7 14:08:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:45.167477 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe102cdb-0e7a-4ce5-8ebf-b64a8df75592 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:45.169986 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc3e0b0-7b96-4654-b827-d6c65f4e3be9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:45.214163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79497d83-3eff-4671-8064-8448ef2db7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:45.214983 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79497d83-3eff-4671-8064-8448ef2db7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:45.215304 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79497d83-3eff-4671-8064-8448ef2db7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:45.216244 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79497d83-3eff-4671-8064-8448ef2db7b9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:45.216913 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79497d83-3eff-4671-8064-8448ef2db7b9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:45.218301 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79497d83-3eff-4671-8064-8448ef2db7b9 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:45.230644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-79497d83-3eff-4671-8064-8448ef2db7b9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 14:08:45.230644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:45.230644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:08:45.230644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:08:45.230644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:08:45.230644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:08:45.230644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:08:45.230644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:45.230644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 14:08:45.230644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:45.230644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:08:45.230644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:45.230644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:45.230644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:45.230644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:45.230644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:45.231923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:45.231923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:45.231923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:45.231923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:45.231923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:45.231923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:45.231923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:45.231923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:08:45.231923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:08:45.231923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:08:45.231923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:08:45.231923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:45.231923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:45.231923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:45.231923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:45.231923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:45.232991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:45.232991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:45.232991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:45.232991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:45.232991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:45.232991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:45.232991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:45.232991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:45.232991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:45.232991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:45.232991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:45.232991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:08:45.232991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:45.232991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 14:08:45.232991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:45.232991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 474/938] 10.222.0.81 () {56 vars in 1132 bytes} [Tue Jun 7 14:08:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:45.237978 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79497d83-3eff-4671-8064-8448ef2db7b9 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:45.303270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc3e0b0-7b96-4654-b827-d6c65f4e3be9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:45.339155 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193414c9-fe03-4b78-b932-d27e5b471f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:45.340097 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193414c9-fe03-4b78-b932-d27e5b471f8e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:45.340560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193414c9-fe03-4b78-b932-d27e5b471f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:45.340983 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193414c9-fe03-4b78-b932-d27e5b471f8e None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:45.341911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193414c9-fe03-4b78-b932-d27e5b471f8e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:45.342402 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193414c9-fe03-4b78-b932-d27e5b471f8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:45.343950 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193414c9-fe03-4b78-b932-d27e5b471f8e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:45.351028 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 465/939] 10.222.0.81 () {56 vars in 1122 bytes} [Tue Jun 7 14:08:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:45.359335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193414c9-fe03-4b78-b932-d27e5b471f8e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:45.361319 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:45.374091 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79497d83-3eff-4671-8064-8448ef2db7b9 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:45.399583 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:45.413064 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78c939-15f4-4880-98f6-83bf98c9ef0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:45.414067 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78c939-15f4-4880-98f6-83bf98c9ef0f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:45.414556 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78c939-15f4-4880-98f6-83bf98c9ef0f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:45.415081 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78c939-15f4-4880-98f6-83bf98c9ef0f None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:45.416371 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78c939-15f4-4880-98f6-83bf98c9ef0f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:45.417114 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78c939-15f4-4880-98f6-83bf98c9ef0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:45.418839 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78c939-15f4-4880-98f6-83bf98c9ef0f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:45.424785 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 475/940] 10.222.0.81 () {56 vars in 1126 bytes} [Tue Jun 7 14:08:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:45.430734 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78c939-15f4-4880-98f6-83bf98c9ef0f None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:45.476463 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193414c9-fe03-4b78-b932-d27e5b471f8e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:45.512658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421449dc-fbc8-4941-bb2e-82c999028938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:45.513635 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421449dc-fbc8-4941-bb2e-82c999028938 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:45.514094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421449dc-fbc8-4941-bb2e-82c999028938 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:45.514540 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421449dc-fbc8-4941-bb2e-82c999028938 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:45.515562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421449dc-fbc8-4941-bb2e-82c999028938 None admin] RBAC: Adding query filter params (interface=public, service_id=b563cb4cf79f444fab6b758b1bde1ced, region_id=RegionOne) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:45.516089 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421449dc-fbc8-4941-bb2e-82c999028938 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:45.517719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421449dc-fbc8-4941-bb2e-82c999028938 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:45.529015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 466/941] 10.222.0.81 () {56 vars in 1251 bytes} [Tue Jun 7 14:08:45 2022] GET /identity/v3/endpoints?service_id=b563cb4cf79f444fab6b758b1bde1ced&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:45.530187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421449dc-fbc8-4941-bb2e-82c999028938 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:45.533334 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78c939-15f4-4880-98f6-83bf98c9ef0f None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:45.569145 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e3917-fbe7-47b0-b032-a52dafe6b669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:45.569384 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e3917-fbe7-47b0-b032-a52dafe6b669 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:45.569609 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e3917-fbe7-47b0-b032-a52dafe6b669 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:45.569842 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e3917-fbe7-47b0-b032-a52dafe6b669 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:45.570703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e3917-fbe7-47b0-b032-a52dafe6b669 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:45.571015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e3917-fbe7-47b0-b032-a52dafe6b669 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:45.572423 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e3917-fbe7-47b0-b032-a52dafe6b669 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:45.582833 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 476/942] 10.222.0.81 () {56 vars in 1126 bytes} [Tue Jun 7 14:08:45 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:45.584269 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e3917-fbe7-47b0-b032-a52dafe6b669 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:45.606209 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:45.635922 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:45.648594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421449dc-fbc8-4941-bb2e-82c999028938 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:45.687060 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9cde4-ce30-4683-a0ca-c52513e70642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:45.688119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9cde4-ce30-4683-a0ca-c52513e70642 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:45.688601 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9cde4-ce30-4683-a0ca-c52513e70642 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:45.689072 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9cde4-ce30-4683-a0ca-c52513e70642 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:45.689533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e3917-fbe7-47b0-b032-a52dafe6b669 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:45.690520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9cde4-ce30-4683-a0ca-c52513e70642 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:45.692370 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9cde4-ce30-4683-a0ca-c52513e70642 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:45.707811 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 467/943] 10.222.0.81 () {56 vars in 1087 bytes} [Tue Jun 7 14:08:45 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:08:45.728299 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbb005-f30c-4978-875c-0f1e7cca57ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:45.728984 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbb005-f30c-4978-875c-0f1e7cca57ec None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:45.729266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbb005-f30c-4978-875c-0f1e7cca57ec None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:45.729540 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbb005-f30c-4978-875c-0f1e7cca57ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:45.730313 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fbb005-f30c-4978-875c-0f1e7cca57ec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:45.731808 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fbb005-f30c-4978-875c-0f1e7cca57ec None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:45.757003 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 477/944] 10.222.0.81 () {60 vars in 1144 bytes} [Tue Jun 7 14:08:45 2022] POST /identity/v3/endpoints => generated 347 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:08:47.002434 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182acb2-6e24-4c6b-856f-3f372ae0a4bd None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:47.002959 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182acb2-6e24-4c6b-856f-3f372ae0a4bd None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:47.002959 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182acb2-6e24-4c6b-856f-3f372ae0a4bd None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:47.004648 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 468/945] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:47.010413 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a924b4d-6853-49bf-99ac-f17a27da62d9 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:47.010841 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a924b4d-6853-49bf-99ac-f17a27da62d9 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:47.011051 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a924b4d-6853-49bf-99ac-f17a27da62d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:47.028970 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6a924b4d-6853-49bf-99ac-f17a27da62d9 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:47.030158 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6a924b4d-6853-49bf-99ac-f17a27da62d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:47.113218 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 478/946] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:08:47.132917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182acb2-6e24-4c6b-856f-3f372ae0a4bd None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:47.248513 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182acb2-6e24-4c6b-856f-3f372ae0a4bd None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:47.279395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:47.284907 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe318c-ee11-4e69-b864-bb79453c11ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:47.285726 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe318c-ee11-4e69-b864-bb79453c11ca None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:47.286672 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe318c-ee11-4e69-b864-bb79453c11ca None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:47.287316 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe318c-ee11-4e69-b864-bb79453c11ca None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:47.298272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effe318c-ee11-4e69-b864-bb79453c11ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:08:47.307134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-effe318c-ee11-4e69-b864-bb79453c11ca None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 14:08:47.307134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:47.307134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:47.307134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:47.307134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:47.307134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:47.307134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:47.307134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:47.307134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:47.307134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:47.307134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:47.307134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:47.307134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:08:47.307134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:08:47.307134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:08:47.307134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:08:47.308124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:08:47.308124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:08:47.308124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:08:47.308124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:08:47.308124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:47.308124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:47.308124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:47.308124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:47.308124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:47.308124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:47.308124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:47.308124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:47.308124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:47.308124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:47.308124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:47.308124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:47.308985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:47.308985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:47.308985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:47.308985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:47.308985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:08:47.308985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:08:47.308985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:08:47.308985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:08:47.308985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:08:47.308985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:08:47.308985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 14:08:47.308985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:47.313318 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 469/947] 10.222.0.81 () {56 vars in 1120 bytes} [Tue Jun 7 14:08:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:47.314687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a924b4d-6853-49bf-99ac-f17a27da62d9 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:47.355569 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:47.419048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a924b4d-6853-49bf-99ac-f17a27da62d9 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:47.454577 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6af86-1ef3-460e-b3d8-59257bb33005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:47.455275 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6af86-1ef3-460e-b3d8-59257bb33005 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:47.455562 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6af86-1ef3-460e-b3d8-59257bb33005 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:47.455836 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6af86-1ef3-460e-b3d8-59257bb33005 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:47.456583 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6af86-1ef3-460e-b3d8-59257bb33005 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:47.456888 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6af86-1ef3-460e-b3d8-59257bb33005 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:47.458374 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6af86-1ef3-460e-b3d8-59257bb33005 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:47.471286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 479/948] 10.222.0.81 () {56 vars in 1115 bytes} [Tue Jun 7 14:08:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:47.472651 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe318c-ee11-4e69-b864-bb79453c11ca None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:47.514984 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2986d-d7e2-4748-95fe-8931ed6694e5 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:47.515367 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2986d-d7e2-4748-95fe-8931ed6694e5 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:47.515568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2986d-d7e2-4748-95fe-8931ed6694e5 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:47.517294 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 480/949] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:08:47.526149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60498a6-1681-4672-837e-e650fce29141 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:47.526438 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60498a6-1681-4672-837e-e650fce29141 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:47.526715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60498a6-1681-4672-837e-e650fce29141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:47.545335 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-b60498a6-1681-4672-837e-e650fce29141 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:47.546570 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b60498a6-1681-4672-837e-e650fce29141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:47.587660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe318c-ee11-4e69-b864-bb79453c11ca None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:47.623114 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 481/950] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:08:47.631602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3054a5-6d6a-4c28-ab97-3ae835eaa152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:47.632668 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3054a5-6d6a-4c28-ab97-3ae835eaa152 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:47.633149 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3054a5-6d6a-4c28-ab97-3ae835eaa152 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:47.633618 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3054a5-6d6a-4c28-ab97-3ae835eaa152 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:47.645868 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60498a6-1681-4672-837e-e650fce29141 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:47.647946 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3054a5-6d6a-4c28-ab97-3ae835eaa152 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:47.650671 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3054a5-6d6a-4c28-ab97-3ae835eaa152 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:47.672736 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:47.673223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-ae3054a5-6d6a-4c28-ab97-3ae835eaa152 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:47.673223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:47.673223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:47.673223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:47.673223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:47.673223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:47.673223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:47.673223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:47.673223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:47.673223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:47.673223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:47.673223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:47.673223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:47.673223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:47.673223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:47.674504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:47.674504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:47.674504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:47.674504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:47.674504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:47.674504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:47.674504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:47.674504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:47.674504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:47.674504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:47.674504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:47.674504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:47.674504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:47.674504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:47.674504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:47.674504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:47.674504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:47.675700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:47.675700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:47.675700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:47.675700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:47.678616 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 470/951] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:47.684104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3054a5-6d6a-4c28-ab97-3ae835eaa152 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:47.735833 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:47.772528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60498a6-1681-4672-837e-e650fce29141 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:47.797872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3054a5-6d6a-4c28-ab97-3ae835eaa152 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:47.811359 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6bac7-c7e8-43ba-8124-c14ab9f8529a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:47.812036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6bac7-c7e8-43ba-8124-c14ab9f8529a None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:47.812324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6bac7-c7e8-43ba-8124-c14ab9f8529a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:47.812604 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6bac7-c7e8-43ba-8124-c14ab9f8529a None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:47.813326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f6bac7-c7e8-43ba-8124-c14ab9f8529a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:47.815015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f6bac7-c7e8-43ba-8124-c14ab9f8529a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:47.816413 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: WARNING keystone.server.flask.common [None req-73f6bac7-c7e8-43ba-8124-c14ab9f8529a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 07 14:08:47.836583 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.sql.core [None req-73f6bac7-c7e8-43ba-8124-c14ab9f8529a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 07 14:08:47.836583 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 07 14:08:47.836583 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [parameters: {'id': 'e172c10f501949058e59c6d1e074d63f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 07 14:08:47.836583 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80234) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 07 14:08:47.837976 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f8e48-fbe6-47a8-a49b-e01fbc6284a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:47.838911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f8e48-fbe6-47a8-a49b-e01fbc6284a3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:47.839452 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f8e48-fbe6-47a8-a49b-e01fbc6284a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:47.839936 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f8e48-fbe6-47a8-a49b-e01fbc6284a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:47.840950 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f8e48-fbe6-47a8-a49b-e01fbc6284a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:47.841459 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f8e48-fbe6-47a8-a49b-e01fbc6284a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:47.843126 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f8e48-fbe6-47a8-a49b-e01fbc6284a3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:47.853519 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-73f6bac7-c7e8-43ba-8124-c14ab9f8529a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e172c10f501949058e59c6d1e074d63f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e172c10f501949058e59c6d1e074d63f. Jun 07 14:08:47.853519 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:47.853519 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 14:08:47.853519 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 14:08:47.853519 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 14:08:47.853519 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 14:08:47.853519 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 14:08:47.853519 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 14:08:47.853519 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 14:08:47.853519 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application conn.query(q) Jun 07 14:08:47.853519 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 14:08:47.853519 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 14:08:47.853519 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 14:08:47.854379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application result.read() Jun 07 14:08:47.854379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 14:08:47.854379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 14:08:47.854379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 14:08:47.854379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 14:08:47.854379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 14:08:47.854379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 14:08:47.854379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 14:08:47.854379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 14:08:47.854379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 07 14:08:47.854379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:47.854379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 07 14:08:47.854379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:47.854379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:47.854379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 07 14:08:47.854379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 07 14:08:47.854379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 07 14:08:47.854379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 07 14:08:47.855417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 14:08:47.855417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application next(self.gen) Jun 07 14:08:47.855417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 07 14:08:47.855417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application yield resource Jun 07 14:08:47.855417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 14:08:47.855417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application next(self.gen) Jun 07 14:08:47.855417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 07 14:08:47.855417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self.session.rollback() Jun 07 14:08:47.855417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 07 14:08:47.855417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self.force_reraise() Jun 07 14:08:47.855417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 07 14:08:47.855417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise self.value Jun 07 14:08:47.855417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 07 14:08:47.855417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 07 14:08:47.855417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 07 14:08:47.855417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application session.commit() Jun 07 14:08:47.855417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 07 14:08:47.856331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 07 14:08:47.856331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 07 14:08:47.856331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._prepare_impl() Jun 07 14:08:47.856331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 07 14:08:47.856331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self.session.flush() Jun 07 14:08:47.856331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 07 14:08:47.856331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._flush(objects) Jun 07 14:08:47.856331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 07 14:08:47.856331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 07 14:08:47.856331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 07 14:08:47.856331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application compat.raise_( Jun 07 14:08:47.856331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 14:08:47.856331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception Jun 07 14:08:47.856331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 07 14:08:47.856331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application flush_context.execute() Jun 07 14:08:47.856331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 07 14:08:47.856331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rec.execute(self) Jun 07 14:08:47.857247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 07 14:08:47.857247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 07 14:08:47.857247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 07 14:08:47.857247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 07 14:08:47.857247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 07 14:08:47.857247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 07 14:08:47.857247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 07 14:08:47.857247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 07 14:08:47.857247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 07 14:08:47.857247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 07 14:08:47.857247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 07 14:08:47.857247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 07 14:08:47.857247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 07 14:08:47.857247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 07 14:08:47.857247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 07 14:08:47.858072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 07 14:08:47.858072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 14:08:47.858072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception Jun 07 14:08:47.858072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 14:08:47.858072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 14:08:47.858072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 14:08:47.858072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 14:08:47.858072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 14:08:47.858072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 14:08:47.858072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 14:08:47.858072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application conn.query(q) Jun 07 14:08:47.858072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 14:08:47.858072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 14:08:47.858072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 14:08:47.858072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application result.read() Jun 07 14:08:47.858072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 14:08:47.858072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 14:08:47.859086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 14:08:47.859086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 14:08:47.859086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 14:08:47.859086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 14:08:47.859086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 14:08:47.859086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 14:08:47.859086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 07 14:08:47.859086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 07 14:08:47.859086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [parameters: {'id': 'e172c10f501949058e59c6d1e074d63f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 07 14:08:47.859086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 07 14:08:47.859086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:47.859086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:08:47.859086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:47.859086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:47.859086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 07 14:08:47.862152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 07 14:08:47.862152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 07 14:08:47.862152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 07 14:08:47.862152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 07 14:08:47.862152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:47.862152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:08:47.862152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:47.862152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:47.862152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:47.862152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:47.862152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:47.862152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:47.862152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:47.862152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:47.862152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:47.862152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:47.862152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:47.864888 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:47.864888 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 07 14:08:47.864888 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 07 14:08:47.864888 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:47.864888 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:47.864888 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 07 14:08:47.864888 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 07 14:08:47.864888 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e172c10f501949058e59c6d1e074d63f. Jun 07 14:08:47.864888 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:47.864888 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 482/952] 10.222.0.81 () {60 vars in 1140 bytes} [Tue Jun 7 14:08:47 2022] POST /identity/v3/projects => generated 262 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:08:47.864888 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6bac7-c7e8-43ba-8124-c14ab9f8529a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:47.866727 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 471/953] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:47.870600 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f8e48-fbe6-47a8-a49b-e01fbc6284a3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:47.986269 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:48.013367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f8e48-fbe6-47a8-a49b-e01fbc6284a3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:48.016485 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:48.023127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6bac7-c7e8-43ba-8124-c14ab9f8529a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:48.063497 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a5af23-e672-4fcb-8464-0ef7b7e9d7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:48.064886 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5af23-e672-4fcb-8464-0ef7b7e9d7aa None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:48.065392 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5af23-e672-4fcb-8464-0ef7b7e9d7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:48.065862 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5af23-e672-4fcb-8464-0ef7b7e9d7aa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:48.069749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d1f09-273b-486c-9c9d-bfcd4ff6dc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:48.070668 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d1f09-273b-486c-9c9d-bfcd4ff6dc7c None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:48.070954 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d1f09-273b-486c-9c9d-bfcd4ff6dc7c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:48.071258 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d1f09-273b-486c-9c9d-bfcd4ff6dc7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:48.075407 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a5af23-e672-4fcb-8464-0ef7b7e9d7aa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:48.076850 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a5af23-e672-4fcb-8464-0ef7b7e9d7aa None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:48.078417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d1f09-273b-486c-9c9d-bfcd4ff6dc7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:48.080083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d1f09-273b-486c-9c9d-bfcd4ff6dc7c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:48.092974 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-96a5af23-e672-4fcb-8464-0ef7b7e9d7aa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 14:08:48.092974 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:48.092974 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:48.092974 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:48.092974 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:48.092974 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:48.092974 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:48.092974 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:48.092974 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:48.092974 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:48.092974 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:48.092974 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:48.092974 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:08:48.092974 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:08:48.092974 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:08:48.092974 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:08:48.094002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:48.094002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:48.094002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:08:48.094002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:48.094002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:08:48.094002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:48.094002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:48.094002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:48.094002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:48.094002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:48.094002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:48.094002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:48.094002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:48.094002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:48.094002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:48.094002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:48.094002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:48.094002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:48.095032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:48.095032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:48.095032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:08:48.095032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:08:48.095032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:08:48.095032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:08:48.095032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:08:48.095032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:08:48.095032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 14:08:48.095032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:48.095032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 472/954] 10.222.0.81 () {56 vars in 1117 bytes} [Tue Jun 7 14:08:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:48.095722 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e13790-52f1-4d69-a619-af5cde9b250a None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:48.096323 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e13790-52f1-4d69-a619-af5cde9b250a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:48.096323 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e13790-52f1-4d69-a619-af5cde9b250a None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:48.097616 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 473/955] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:48.099678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-fb3d1f09-273b-486c-9c9d-bfcd4ff6dc7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:48.099678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:48.099678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:48.099678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:48.099678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:48.099678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:48.099678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:48.099678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:48.099678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:48.099678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:48.099678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:48.099678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:48.099678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:08:48.099678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:08:48.099678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:08:48.100663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:08:48.100663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:48.100663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:48.100663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:48.100663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:48.100663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:48.100663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:48.100663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:48.100663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:48.100663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:48.100663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:48.100663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:48.100663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:48.100663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:48.100663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:48.100663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:48.100663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:48.103056 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:08:48.103056 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:08:48.103056 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:08:48.103056 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:08:48.103056 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:08:48.103056 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:08:48.103056 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:48.103056 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:48.103056 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 483/956] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:48.104222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef95b15-1b3f-4af8-8d10-63c6960d3ec9 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:48.104222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef95b15-1b3f-4af8-8d10-63c6960d3ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:48.104222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef95b15-1b3f-4af8-8d10-63c6960d3ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:48.105213 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e13790-52f1-4d69-a619-af5cde9b250a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:48.122487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-8ef95b15-1b3f-4af8-8d10-63c6960d3ec9 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:48.124017 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8ef95b15-1b3f-4af8-8d10-63c6960d3ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:48.203754 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 484/957] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:08:48.204598 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef95b15-1b3f-4af8-8d10-63c6960d3ec9 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:48.229548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e13790-52f1-4d69-a619-af5cde9b250a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:48.269110 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62999f6-6de2-462d-ac56-00b21c59a936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:48.270074 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62999f6-6de2-462d-ac56-00b21c59a936 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:48.270360 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62999f6-6de2-462d-ac56-00b21c59a936 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:48.270673 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62999f6-6de2-462d-ac56-00b21c59a936 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:48.271722 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62999f6-6de2-462d-ac56-00b21c59a936 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:48.272035 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62999f6-6de2-462d-ac56-00b21c59a936 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:48.273479 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62999f6-6de2-462d-ac56-00b21c59a936 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:48.278236 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:48.293177 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 474/958] 10.222.0.81 () {56 vars in 1149 bytes} [Tue Jun 7 14:08:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:48.294022 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62999f6-6de2-462d-ac56-00b21c59a936 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:48.316555 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef95b15-1b3f-4af8-8d10-63c6960d3ec9 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:48.324179 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:48.351961 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204996bc-f8b7-4efd-8efa-2c01da5af0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:48.352985 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204996bc-f8b7-4efd-8efa-2c01da5af0cb None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:48.353388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204996bc-f8b7-4efd-8efa-2c01da5af0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:48.353750 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204996bc-f8b7-4efd-8efa-2c01da5af0cb None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:48.354505 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204996bc-f8b7-4efd-8efa-2c01da5af0cb None admin] RBAC: Adding query filter params (name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:48.354805 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204996bc-f8b7-4efd-8efa-2c01da5af0cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:48.356398 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204996bc-f8b7-4efd-8efa-2c01da5af0cb None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:48.385624 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 485/959] 10.222.0.81 () {56 vars in 1112 bytes} [Tue Jun 7 14:08:48 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:48.387140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204996bc-f8b7-4efd-8efa-2c01da5af0cb None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:48.409004 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62999f6-6de2-462d-ac56-00b21c59a936 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:48.449338 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2f8cf-0bfc-4dcb-ac81-f6ff844ae422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:48.450176 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2f8cf-0bfc-4dcb-ac81-f6ff844ae422 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:48.450456 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2f8cf-0bfc-4dcb-ac81-f6ff844ae422 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:48.450725 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2f8cf-0bfc-4dcb-ac81-f6ff844ae422 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:48.451442 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2f8cf-0bfc-4dcb-ac81-f6ff844ae422 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:48.452853 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2f8cf-0bfc-4dcb-ac81-f6ff844ae422 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:48.465785 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-ceb2f8cf-0bfc-4dcb-ac81-f6ff844ae422 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:08:48.465785 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:48.465785 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:08:48.465785 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:08:48.465785 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:08:48.465785 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:08:48.465785 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:08:48.465785 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:48.465785 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:08:48.465785 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:48.465785 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:08:48.465785 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:48.465785 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:48.465785 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:48.465785 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:48.465785 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:48.465785 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:48.468178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:48.468178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:48.468178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:48.468178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:48.468178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:48.468178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:48.468178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:08:48.468178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:08:48.468178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:08:48.468178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:08:48.468178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:48.468178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:48.468178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:48.468178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:48.468178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:48.468178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:48.470064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:48.470064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:48.470064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:48.470064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:48.470064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:48.470064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:48.470064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:48.470064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:48.470064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:48.470064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:48.470064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:08:48.470064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:48.470064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:08:48.470064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:48.472469 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 475/960] 10.222.0.81 () {56 vars in 1126 bytes} [Tue Jun 7 14:08:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:48.474581 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ab270-72c1-469b-bf48-96d40eaf813c None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:48.474868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ab270-72c1-469b-bf48-96d40eaf813c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:48.475142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ab270-72c1-469b-bf48-96d40eaf813c None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:48.476758 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 476/961] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:48.477330 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ab270-72c1-469b-bf48-96d40eaf813c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:48.517940 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204996bc-f8b7-4efd-8efa-2c01da5af0cb None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:48.547557 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:48.560820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d8e48-9b96-420f-aa4d-dad8c1628c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:48.561490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d8e48-9b96-420f-aa4d-dad8c1628c79 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:48.561774 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d8e48-9b96-420f-aa4d-dad8c1628c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:48.562051 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d8e48-9b96-420f-aa4d-dad8c1628c79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:48.577302 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:48.595969 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ab270-72c1-469b-bf48-96d40eaf813c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:48.643956 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d8e48-9b96-420f-aa4d-dad8c1628c79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:48.644891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd7d87-7882-4cf6-9a64-4b61c8d207a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:48.646115 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd7d87-7882-4cf6-9a64-4b61c8d207a7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:48.646342 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d8e48-9b96-420f-aa4d-dad8c1628c79 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:48.646454 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd7d87-7882-4cf6-9a64-4b61c8d207a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:48.646698 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd7d87-7882-4cf6-9a64-4b61c8d207a7 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:48.647494 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfd7d87-7882-4cf6-9a64-4b61c8d207a7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:48.647804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfd7d87-7882-4cf6-9a64-4b61c8d207a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:48.649174 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfd7d87-7882-4cf6-9a64-4b61c8d207a7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:48.654744 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 477/962] 10.222.0.81 () {56 vars in 1122 bytes} [Tue Jun 7 14:08:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:48.656151 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbf0c1-2ea9-44f9-a739-d6b72148edc0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:48.656437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbf0c1-2ea9-44f9-a739-d6b72148edc0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:48.656715 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbf0c1-2ea9-44f9-a739-d6b72148edc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:48.674821 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2edbf0c1-2ea9-44f9-a739-d6b72148edc0 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:48.676057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2edbf0c1-2ea9-44f9-a739-d6b72148edc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:48.748330 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 478/963] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:08:48.749431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbf0c1-2ea9-44f9-a739-d6b72148edc0 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:48.776699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 486/964] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:08:48 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:08:48.777769 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d8e48-9b96-420f-aa4d-dad8c1628c79 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:48.808670 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:48.837476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:48.865046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbf0c1-2ea9-44f9-a739-d6b72148edc0 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:48.897791 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d8e48-9b96-420f-aa4d-dad8c1628c79 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:48.901251 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5305ec1-b3aa-4619-9d13-8fc06fd3f5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:48.901921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5305ec1-b3aa-4619-9d13-8fc06fd3f5a1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:48.902204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5305ec1-b3aa-4619-9d13-8fc06fd3f5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:48.902477 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5305ec1-b3aa-4619-9d13-8fc06fd3f5a1 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:48.903213 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5305ec1-b3aa-4619-9d13-8fc06fd3f5a1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:48.903636 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5305ec1-b3aa-4619-9d13-8fc06fd3f5a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:48.904998 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5305ec1-b3aa-4619-9d13-8fc06fd3f5a1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:48.912184 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 479/965] 10.222.0.81 () {56 vars in 1122 bytes} [Tue Jun 7 14:08:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:48.913359 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5305ec1-b3aa-4619-9d13-8fc06fd3f5a1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:48.940181 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3c701-4584-4d15-8e43-b4329a513d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:48.941126 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3c701-4584-4d15-8e43-b4329a513d16 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:48.941507 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3c701-4584-4d15-8e43-b4329a513d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:48.941820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3c701-4584-4d15-8e43-b4329a513d16 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:48.942673 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3c701-4584-4d15-8e43-b4329a513d16 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:48.944427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3c701-4584-4d15-8e43-b4329a513d16 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:48.956320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-3dc3c701-4584-4d15-8e43-b4329a513d16 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 14:08:48.956320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:48.956320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:08:48.956320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:08:48.956320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:08:48.956320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:08:48.956320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:08:48.956320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:48.956320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 14:08:48.956320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:48.956320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:08:48.956320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:48.956320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:48.956320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:48.956320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:48.956320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:48.956320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:48.957567 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:48.957567 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:48.957567 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:48.957567 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:48.957567 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:48.957567 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:48.957567 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:08:48.957567 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:08:48.957567 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:08:48.957567 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:08:48.957567 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:48.957567 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:48.957567 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:48.957567 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:48.957567 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:48.957567 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:48.958657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:48.958657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:48.958657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:48.958657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:48.958657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:48.958657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:48.958657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:48.958657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:48.958657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:48.958657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:48.958657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:08:48.958657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:48.958657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 14:08:48.958657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:48.960835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 487/966] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:48.966237 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3c701-4584-4d15-8e43-b4329a513d16 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:49.021400 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5305ec1-b3aa-4619-9d13-8fc06fd3f5a1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:49.058205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985d850-9900-4a47-ab41-01ed7a035ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:49.059002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985d850-9900-4a47-ab41-01ed7a035ffc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:49.059324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985d850-9900-4a47-ab41-01ed7a035ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:49.059589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985d850-9900-4a47-ab41-01ed7a035ffc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:49.069921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e985d850-9900-4a47-ab41-01ed7a035ffc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:49.071400 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e985d850-9900-4a47-ab41-01ed7a035ffc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:49.072111 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:49.074700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:49.082501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3c701-4584-4d15-8e43-b4329a513d16 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:49.091204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-e985d850-9900-4a47-ab41-01ed7a035ffc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:49.091204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:49.091204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:49.091204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:49.091204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:49.091204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:49.091204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:49.091204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:49.091204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:49.091204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:49.091204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:49.091204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:49.091204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:08:49.091204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:08:49.091204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:08:49.092259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:08:49.092259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:49.092259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:49.092259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:49.092259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:49.092259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:49.092259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:49.092259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:49.092259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:49.092259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:49.092259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:49.092259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:49.092259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:49.092259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:49.092259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:49.092259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:49.092259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:49.093201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:08:49.093201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:08:49.093201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:08:49.093201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:49.093201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 480/967] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:49.093201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985d850-9900-4a47-ab41-01ed7a035ffc None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:49.124719 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430db86-558b-4ec9-85ce-d95338c133f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:49.125415 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430db86-558b-4ec9-85ce-d95338c133f3 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:49.125696 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430db86-558b-4ec9-85ce-d95338c133f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:49.125973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430db86-558b-4ec9-85ce-d95338c133f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:49.126707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430db86-558b-4ec9-85ce-d95338c133f3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:49.128105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430db86-558b-4ec9-85ce-d95338c133f3 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:49.159427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 488/968] 10.222.0.81 () {60 vars in 1144 bytes} [Tue Jun 7 14:08:48 2022] POST /identity/v3/endpoints => generated 349 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:08:49.161041 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430db86-558b-4ec9-85ce-d95338c133f3 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:49.243895 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985d850-9900-4a47-ab41-01ed7a035ffc None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:49.301454 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430db86-558b-4ec9-85ce-d95338c133f3 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:49.307378 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c40575-1547-49a7-b2b0-716a923f57bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:49.308341 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c40575-1547-49a7-b2b0-716a923f57bd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:49.308767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c40575-1547-49a7-b2b0-716a923f57bd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:49.309260 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c40575-1547-49a7-b2b0-716a923f57bd None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:49.310146 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c40575-1547-49a7-b2b0-716a923f57bd None admin] RBAC: Adding query filter params (name=compute) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:49.311321 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c40575-1547-49a7-b2b0-716a923f57bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:49.312898 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c40575-1547-49a7-b2b0-716a923f57bd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:49.325755 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 481/969] 10.222.0.81 () {56 vars in 1112 bytes} [Tue Jun 7 14:08:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:49.327762 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c40575-1547-49a7-b2b0-716a923f57bd None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:49.347458 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915f082-5add-4df8-9e88-219f95366563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:49.348195 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915f082-5add-4df8-9e88-219f95366563 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:49.348702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915f082-5add-4df8-9e88-219f95366563 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:49.349586 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915f082-5add-4df8-9e88-219f95366563 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:49.350439 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7915f082-5add-4df8-9e88-219f95366563 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:49.350845 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7915f082-5add-4df8-9e88-219f95366563 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:49.352665 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7915f082-5add-4df8-9e88-219f95366563 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:49.367125 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 489/970] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:49.373270 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915f082-5add-4df8-9e88-219f95366563 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:49.377802 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:49.406310 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:49.478116 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c40575-1547-49a7-b2b0-716a923f57bd None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:49.499574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915f082-5add-4df8-9e88-219f95366563 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:49.526602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e041cb7-a774-40a7-9148-e828c9fce3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:49.527546 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e041cb7-a774-40a7-9148-e828c9fce3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:49.527845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e041cb7-a774-40a7-9148-e828c9fce3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:49.528127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e041cb7-a774-40a7-9148-e828c9fce3a2 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:49.529275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e041cb7-a774-40a7-9148-e828c9fce3a2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:49.529589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e041cb7-a774-40a7-9148-e828c9fce3a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:49.531545 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e041cb7-a774-40a7-9148-e828c9fce3a2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:49.538400 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 482/971] 10.222.0.81 () {56 vars in 1112 bytes} [Tue Jun 7 14:08:49 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:49.546621 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf48076-fbf3-40da-9544-e66fe66e4ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:49.547491 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf48076-fbf3-40da-9544-e66fe66e4ade None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:49.547777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf48076-fbf3-40da-9544-e66fe66e4ade None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:49.548053 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf48076-fbf3-40da-9544-e66fe66e4ade None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:49.556152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf48076-fbf3-40da-9544-e66fe66e4ade None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:49.557759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf48076-fbf3-40da-9544-e66fe66e4ade None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:49.574126 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-baf48076-fbf3-40da-9544-e66fe66e4ade None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:49.574126 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:49.574126 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:49.574126 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:49.574126 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:49.574126 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:49.574126 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:49.574126 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:49.574126 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:49.574126 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:49.574126 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:49.574126 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:49.574126 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:08:49.574126 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:08:49.574126 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:08:49.575326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:08:49.575326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:49.575326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:49.575326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:49.575326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:49.575326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:49.575326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:49.575326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:49.575326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:49.575326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:49.575326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:49.575326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:49.575326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:49.575326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:49.575326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:49.575326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:49.575326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:49.576470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:08:49.576470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:08:49.576470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:08:49.576470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:08:49.576470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:08:49.576470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:08:49.576470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:08:49.576470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:49.576470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 490/972] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:49.583866 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e041cb7-a774-40a7-9148-e828c9fce3a2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:49.704592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e041cb7-a774-40a7-9148-e828c9fce3a2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:49.730795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:49.747211 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46856a1-7f91-43f2-bc66-ed6d188f1589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:49.747929 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46856a1-7f91-43f2-bc66-ed6d188f1589 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:49.748501 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46856a1-7f91-43f2-bc66-ed6d188f1589 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:49.749135 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46856a1-7f91-43f2-bc66-ed6d188f1589 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:49.750193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46856a1-7f91-43f2-bc66-ed6d188f1589 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:49.750982 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46856a1-7f91-43f2-bc66-ed6d188f1589 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:49.752556 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46856a1-7f91-43f2-bc66-ed6d188f1589 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:49.766104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 483/973] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:08:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:49.772322 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf48076-fbf3-40da-9544-e66fe66e4ade None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:49.842346 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:49.883655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf48076-fbf3-40da-9544-e66fe66e4ade None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:49.920562 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2a7a1-813e-48aa-ab5e-14a039b5163d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:49.921224 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2a7a1-813e-48aa-ab5e-14a039b5163d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:49.921511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2a7a1-813e-48aa-ab5e-14a039b5163d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:49.921786 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2a7a1-813e-48aa-ab5e-14a039b5163d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:49.922616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c2a7a1-813e-48aa-ab5e-14a039b5163d None admin] RBAC: Adding query filter params (role.id=855dcf6914224d5c907aa41ec8c2e0c4, scope.project.id=f67bad575f8b4a47bee7c314497cb2fe, user.id=d00b8bc311234e6da47b365ac8a7183a) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:49.922925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c2a7a1-813e-48aa-ab5e-14a039b5163d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:49.924562 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c2a7a1-813e-48aa-ab5e-14a039b5163d None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:49.931760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 491/974] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:08:49 2022] GET /identity/v3/role_assignments?user.id=d00b8bc311234e6da47b365ac8a7183a&scope.project.id=f67bad575f8b4a47bee7c314497cb2fe&role.id=855dcf6914224d5c907aa41ec8c2e0c4 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:50.781050 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49065597-242b-4343-853a-9ab9be004f12 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:50.781573 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49065597-242b-4343-853a-9ab9be004f12 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:50.781651 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49065597-242b-4343-853a-9ab9be004f12 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:50.783163 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 484/975] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:50.788910 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92d563-ab19-4c30-96ce-15ce693f5b49 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:50.789375 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92d563-ab19-4c30-96ce-15ce693f5b49 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:50.789456 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92d563-ab19-4c30-96ce-15ce693f5b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:50.807177 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-cd92d563-ab19-4c30-96ce-15ce693f5b49 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:50.808449 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-cd92d563-ab19-4c30-96ce-15ce693f5b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:50.885311 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 492/976] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:08:50.912501 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49065597-242b-4343-853a-9ab9be004f12 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:51.016846 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49065597-242b-4343-853a-9ab9be004f12 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:51.053906 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b429d-8568-4dff-a8b3-586058c0fe61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:51.054911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b429d-8568-4dff-a8b3-586058c0fe61 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:51.055415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b429d-8568-4dff-a8b3-586058c0fe61 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:51.055906 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b429d-8568-4dff-a8b3-586058c0fe61 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:51.056823 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b429d-8568-4dff-a8b3-586058c0fe61 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:51.058435 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b429d-8568-4dff-a8b3-586058c0fe61 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:51.059969 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: WARNING keystone.server.flask.common [None req-3a3b429d-8568-4dff-a8b3-586058c0fe61 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 07 14:08:51.073975 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.sql.core [None req-3a3b429d-8568-4dff-a8b3-586058c0fe61 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 07 14:08:51.073975 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 07 14:08:51.073975 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [parameters: {'user_id': 'c61cf7000c1b4f9f97bc45a6869c0fe5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 07 14:08:51.073975 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80233) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 07 14:08:51.082201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-3a3b429d-8568-4dff-a8b3-586058c0fe61 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 07 14:08:51.082201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:51.082201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 14:08:51.082201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 14:08:51.082201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 14:08:51.082201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 14:08:51.082201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 14:08:51.082201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 14:08:51.082201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 14:08:51.082201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application conn.query(q) Jun 07 14:08:51.082201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 14:08:51.082201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 14:08:51.082201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 14:08:51.082201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application result.read() Jun 07 14:08:51.082201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 14:08:51.083704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 14:08:51.083704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 14:08:51.083704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 14:08:51.083704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 14:08:51.083704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 14:08:51.083704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 14:08:51.083704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 14:08:51.083704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 07 14:08:51.083704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:51.083704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 07 14:08:51.083704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:51.083704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:51.083704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 07 14:08:51.083704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 07 14:08:51.083704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 07 14:08:51.083704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 07 14:08:51.083704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 14:08:51.083704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application next(self.gen) Jun 07 14:08:51.084996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 07 14:08:51.084996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application yield resource Jun 07 14:08:51.084996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 14:08:51.084996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application next(self.gen) Jun 07 14:08:51.084996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 07 14:08:51.084996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.session.rollback() Jun 07 14:08:51.084996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 07 14:08:51.084996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.force_reraise() Jun 07 14:08:51.084996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 07 14:08:51.084996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise self.value Jun 07 14:08:51.084996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 07 14:08:51.084996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 07 14:08:51.084996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 07 14:08:51.084996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application session.commit() Jun 07 14:08:51.084996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 07 14:08:51.084996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 07 14:08:51.084996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 07 14:08:51.086484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._prepare_impl() Jun 07 14:08:51.086484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 07 14:08:51.086484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.session.flush() Jun 07 14:08:51.086484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 07 14:08:51.086484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._flush(objects) Jun 07 14:08:51.086484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 07 14:08:51.086484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 07 14:08:51.086484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 07 14:08:51.086484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application compat.raise_( Jun 07 14:08:51.086484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 14:08:51.086484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception Jun 07 14:08:51.086484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 07 14:08:51.086484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application flush_context.execute() Jun 07 14:08:51.086484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 07 14:08:51.086484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rec.execute(self) Jun 07 14:08:51.086484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 07 14:08:51.086484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 07 14:08:51.088794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 07 14:08:51.088794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 07 14:08:51.088794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 07 14:08:51.088794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 07 14:08:51.088794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 07 14:08:51.088794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 07 14:08:51.088794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 07 14:08:51.088794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 07 14:08:51.088794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 07 14:08:51.088794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 07 14:08:51.088794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 07 14:08:51.088794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 07 14:08:51.088794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 07 14:08:51.088794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 07 14:08:51.088794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 14:08:51.088794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception Jun 07 14:08:51.090784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 14:08:51.090784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 14:08:51.090784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 14:08:51.090784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 14:08:51.090784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 14:08:51.090784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 14:08:51.090784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 14:08:51.090784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application conn.query(q) Jun 07 14:08:51.090784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 14:08:51.090784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 14:08:51.090784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 14:08:51.090784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application result.read() Jun 07 14:08:51.090784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 14:08:51.090784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 14:08:51.090784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 14:08:51.090784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 14:08:51.092885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 14:08:51.092885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 14:08:51.092885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 14:08:51.092885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 14:08:51.092885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 07 14:08:51.092885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 07 14:08:51.092885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c61cf7000c1b4f9f97bc45a6869c0fe5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 07 14:08:51.092885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 07 14:08:51.092885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:51.092885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:08:51.092885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:51.092885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:51.092885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:51.092885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:51.092885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:51.094814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:51.094814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:51.094814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:51.094814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:51.094814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:51.094814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:51.094814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:51.094814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 07 14:08:51.094814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 07 14:08:51.094814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:51.094814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:51.094814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:08:51.094814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:51.094814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:08:51.094814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:51.094814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 07 14:08:51.094814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 07 14:08:51.098376 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 07 14:08:51.098376 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 07 14:08:51.098376 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 07 14:08:51.098376 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 07 14:08:51.098376 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 07 14:08:51.098376 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:51.100371 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 485/977] 10.222.0.81 () {60 vars in 1131 bytes} [Tue Jun 7 14:08:50 2022] POST /identity/v3/users => generated 160 bytes in 188 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:51.107316 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92d563-ab19-4c30-96ce-15ce693f5b49 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:51.187637 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:51.235922 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92d563-ab19-4c30-96ce-15ce693f5b49 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:51.277201 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9a0b8-be83-4ebb-a511-fb188e720f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:51.278010 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9a0b8-be83-4ebb-a511-fb188e720f94 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:51.278298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9a0b8-be83-4ebb-a511-fb188e720f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:51.278577 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9a0b8-be83-4ebb-a511-fb188e720f94 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:51.285950 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a9a0b8-be83-4ebb-a511-fb188e720f94 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:51.288118 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a9a0b8-be83-4ebb-a511-fb188e720f94 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:51.303426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-61a9a0b8-be83-4ebb-a511-fb188e720f94 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 14:08:51.303426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:51.303426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:51.303426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:51.303426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:51.303426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:51.303426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:51.303426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:51.303426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:51.303426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:51.303426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:51.303426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:51.303426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:08:51.303426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:08:51.303426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:08:51.303426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:08:51.304489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:51.304489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:51.304489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:08:51.304489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:51.304489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:08:51.304489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:08:51.304489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:51.304489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:51.304489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:51.304489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:51.304489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:51.304489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:51.304489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:51.304489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:51.304489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:51.304489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:51.304489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:51.304489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:51.305885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:51.305885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:51.305885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:08:51.305885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:08:51.305885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:08:51.305885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:08:51.305885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:08:51.305885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:08:51.305885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 14:08:51.305885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:51.305885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 493/978] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:08:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:51.312703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b429d-8568-4dff-a8b3-586058c0fe61 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:51.362700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:51.429284 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b429d-8568-4dff-a8b3-586058c0fe61 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:51.464535 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c8ece-b318-43e1-9e60-fa09af586fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:51.465221 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c8ece-b318-43e1-9e60-fa09af586fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:51.465508 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c8ece-b318-43e1-9e60-fa09af586fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:51.465783 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c8ece-b318-43e1-9e60-fa09af586fa5 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:51.466587 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883c8ece-b318-43e1-9e60-fa09af586fa5 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:51.466890 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883c8ece-b318-43e1-9e60-fa09af586fa5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:51.468555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883c8ece-b318-43e1-9e60-fa09af586fa5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:51.508940 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 486/979] 10.222.0.81 () {56 vars in 1101 bytes} [Tue Jun 7 14:08:51 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:51.516098 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2d25e-1af5-492e-9de8-30339538f343 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:51.516385 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2d25e-1af5-492e-9de8-30339538f343 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:51.516656 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2d25e-1af5-492e-9de8-30339538f343 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:51.518354 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 494/980] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:51.524964 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e95d04-9efd-4ef8-a8aa-9f3bb1ac7499 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:51.525296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e95d04-9efd-4ef8-a8aa-9f3bb1ac7499 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:51.525531 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e95d04-9efd-4ef8-a8aa-9f3bb1ac7499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:51.546308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-26e95d04-9efd-4ef8-a8aa-9f3bb1ac7499 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:51.547947 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-26e95d04-9efd-4ef8-a8aa-9f3bb1ac7499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:51.631423 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 487/981] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:08:51.651569 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2d25e-1af5-492e-9de8-30339538f343 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:51.768587 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2d25e-1af5-492e-9de8-30339538f343 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:51.787707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:51.807264 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97bcf19-dd00-48c6-a115-4c2461f1f831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:51.808276 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bcf19-dd00-48c6-a115-4c2461f1f831 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:51.808662 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bcf19-dd00-48c6-a115-4c2461f1f831 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:51.809036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bcf19-dd00-48c6-a115-4c2461f1f831 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:51.809796 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97bcf19-dd00-48c6-a115-4c2461f1f831 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:51.811276 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97bcf19-dd00-48c6-a115-4c2461f1f831 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:51.824886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-d97bcf19-dd00-48c6-a115-4c2461f1f831 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:08:51.824886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:51.824886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:08:51.824886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:08:51.824886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:08:51.824886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:08:51.824886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:08:51.824886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:51.824886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:08:51.824886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:51.824886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:08:51.824886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:51.824886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:51.824886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:51.824886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:51.824886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:51.824886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:51.825942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:51.825942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:51.825942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:51.825942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:51.825942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:51.825942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:51.825942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:08:51.825942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:08:51.825942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:08:51.825942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:08:51.825942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:51.825942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:51.825942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:51.825942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:51.825942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:51.825942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:51.827113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:51.827113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:51.827113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:51.827113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:51.827113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:51.827113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:51.827113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:51.827113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:51.827113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:51.827113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:51.827113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:08:51.827113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:51.827113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:08:51.827113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:51.827113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 495/982] 10.222.0.81 () {56 vars in 1126 bytes} [Tue Jun 7 14:08:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:51.832665 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e95d04-9efd-4ef8-a8aa-9f3bb1ac7499 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:51.868675 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:51.942109 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e95d04-9efd-4ef8-a8aa-9f3bb1ac7499 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:51.980481 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de788f27-3396-45b5-b1b1-bf9c04f540fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:51.981158 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de788f27-3396-45b5-b1b1-bf9c04f540fa None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:51.981441 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de788f27-3396-45b5-b1b1-bf9c04f540fa None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:51.981716 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de788f27-3396-45b5-b1b1-bf9c04f540fa None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:51.982437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de788f27-3396-45b5-b1b1-bf9c04f540fa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:51.982741 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de788f27-3396-45b5-b1b1-bf9c04f540fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:51.984184 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de788f27-3396-45b5-b1b1-bf9c04f540fa None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:51.990023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 488/983] 10.222.0.81 () {56 vars in 1122 bytes} [Tue Jun 7 14:08:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:51.995820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97bcf19-dd00-48c6-a115-4c2461f1f831 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:52.044688 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15660c-a120-482a-aaf1-99035945da42 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:52.045036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15660c-a120-482a-aaf1-99035945da42 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:52.045259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15660c-a120-482a-aaf1-99035945da42 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:52.046638 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 489/984] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:52.051446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d083148-1b95-4f7b-b806-315603a221be None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:52.052418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d083148-1b95-4f7b-b806-315603a221be None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:52.052899 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d083148-1b95-4f7b-b806-315603a221be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:52.072086 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1d083148-1b95-4f7b-b806-315603a221be None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:52.073278 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1d083148-1b95-4f7b-b806-315603a221be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:52.121869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97bcf19-dd00-48c6-a115-4c2461f1f831 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:52.146034 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 490/985] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:08:52.160725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07410428-43bb-48d8-aaea-b29672435b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:52.164325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07410428-43bb-48d8-aaea-b29672435b6a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:52.164325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07410428-43bb-48d8-aaea-b29672435b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:52.164325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07410428-43bb-48d8-aaea-b29672435b6a None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:52.164325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07410428-43bb-48d8-aaea-b29672435b6a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:52.164325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07410428-43bb-48d8-aaea-b29672435b6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:52.164711 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07410428-43bb-48d8-aaea-b29672435b6a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:52.168810 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d083148-1b95-4f7b-b806-315603a221be None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:52.174499 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 496/986] 10.222.0.81 () {56 vars in 1122 bytes} [Tue Jun 7 14:08:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:52.176764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07410428-43bb-48d8-aaea-b29672435b6a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:52.199172 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:52.227396 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:52.282515 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d083148-1b95-4f7b-b806-315603a221be None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:52.283868 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07410428-43bb-48d8-aaea-b29672435b6a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:52.318497 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8a926-535b-4995-9685-772e7c587f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:52.318821 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f1fc0-2f55-446f-84ab-3f1ebff16855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:52.319160 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8a926-535b-4995-9685-772e7c587f79 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:52.319469 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8a926-535b-4995-9685-772e7c587f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:52.319562 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f1fc0-2f55-446f-84ab-3f1ebff16855 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:52.319736 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8a926-535b-4995-9685-772e7c587f79 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:52.319842 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f1fc0-2f55-446f-84ab-3f1ebff16855 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:52.320015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f1fc0-2f55-446f-84ab-3f1ebff16855 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:52.320446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d8a926-535b-4995-9685-772e7c587f79 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:52.320837 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f1fc0-2f55-446f-84ab-3f1ebff16855 None admin] RBAC: Adding query filter params (interface=admin, service_id=b563cb4cf79f444fab6b758b1bde1ced, region_id=RegionOne) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:52.321150 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f1fc0-2f55-446f-84ab-3f1ebff16855 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:52.321811 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d8a926-535b-4995-9685-772e7c587f79 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:52.322492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f1fc0-2f55-446f-84ab-3f1ebff16855 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:52.327963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 497/987] 10.222.0.81 () {56 vars in 1249 bytes} [Tue Jun 7 14:08:52 2022] GET /identity/v3/endpoints?service_id=b563cb4cf79f444fab6b758b1bde1ced&interface=admin®ion_id=RegionOne => generated 193 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:52.329409 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6fd6a-8e96-47db-a7bb-dedaa5f3dff7 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:52.329766 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6fd6a-8e96-47db-a7bb-dedaa5f3dff7 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:52.330018 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6fd6a-8e96-47db-a7bb-dedaa5f3dff7 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:52.331262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 498/988] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:52.333939 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6fd6a-8e96-47db-a7bb-dedaa5f3dff7 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:52.343256 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 491/989] 10.222.0.81 () {60 vars in 1141 bytes} [Tue Jun 7 14:08:52 2022] POST /identity/v3/services => generated 242 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:08:52.343256 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2835fc7-7a40-451e-94b3-6a004abd8010 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:52.343256 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2835fc7-7a40-451e-94b3-6a004abd8010 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:52.343256 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2835fc7-7a40-451e-94b3-6a004abd8010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:52.360217 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c2835fc7-7a40-451e-94b3-6a004abd8010 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:52.361468 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c2835fc7-7a40-451e-94b3-6a004abd8010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:52.463340 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6fd6a-8e96-47db-a7bb-dedaa5f3dff7 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:52.463540 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 492/990] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:52 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:08:52.479723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2835fc7-7a40-451e-94b3-6a004abd8010 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:52.480970 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:52.495065 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:52.499753 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbeba0-7e08-4ca0-acf9-a445e79ea575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:52.500747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbeba0-7e08-4ca0-acf9-a445e79ea575 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:52.500747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbeba0-7e08-4ca0-acf9-a445e79ea575 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:52.501033 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbeba0-7e08-4ca0-acf9-a445e79ea575 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:52.501736 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cbeba0-7e08-4ca0-acf9-a445e79ea575 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:52.503250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cbeba0-7e08-4ca0-acf9-a445e79ea575 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:52.515258 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 499/991] 10.222.0.81 () {56 vars in 1087 bytes} [Tue Jun 7 14:08:52 2022] GET /identity/v3/services => generated 1727 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:08:52.612075 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2835fc7-7a40-451e-94b3-6a004abd8010 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:52.650334 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c34721a-30ac-407b-9918-e5aa61fcf1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:52.651147 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34721a-30ac-407b-9918-e5aa61fcf1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:52.651463 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34721a-30ac-407b-9918-e5aa61fcf1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:52.651744 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34721a-30ac-407b-9918-e5aa61fcf1b2 None admin] PATH_INFO: `/v3/services/image` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:52.652410 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c34721a-30ac-407b-9918-e5aa61fcf1b2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:52.653820 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c34721a-30ac-407b-9918-e5aa61fcf1b2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:52.666896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-0c34721a-30ac-407b-9918-e5aa61fcf1b2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 14:08:52.666896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:52.666896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:08:52.666896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:08:52.666896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:08:52.666896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:08:52.666896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:08:52.666896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:52.666896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 14:08:52.666896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:52.666896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:08:52.666896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:52.666896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:52.666896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:52.666896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:52.666896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:52.666896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:52.668214 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:52.668214 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:52.668214 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:52.668214 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:52.668214 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:52.668214 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:52.668214 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:08:52.668214 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:08:52.668214 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:08:52.668214 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:08:52.668214 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:52.668214 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:52.668214 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:52.668214 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:52.668214 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:52.668214 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:52.669254 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:52.669254 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:52.669254 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:52.669254 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:52.669254 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:52.669254 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:52.669254 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:52.669254 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:52.669254 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:52.669254 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:52.669254 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:08:52.669254 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:52.669254 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 14:08:52.669254 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:52.669254 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 493/992] 10.222.0.81 () {56 vars in 1105 bytes} [Tue Jun 7 14:08:52 2022] GET /identity/v3/services/image => generated 86 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:08:52.676283 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbeba0-7e08-4ca0-acf9-a445e79ea575 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:52.784129 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbeba0-7e08-4ca0-acf9-a445e79ea575 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:52.818522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d69c64-b35a-4caa-840f-956fe8eb4dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:52.819164 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d69c64-b35a-4caa-840f-956fe8eb4dce None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:52.819622 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d69c64-b35a-4caa-840f-956fe8eb4dce None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:52.819899 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d69c64-b35a-4caa-840f-956fe8eb4dce None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:52.820603 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d69c64-b35a-4caa-840f-956fe8eb4dce None admin] RBAC: Adding query filter params (name=image) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:52.820903 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d69c64-b35a-4caa-840f-956fe8eb4dce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:52.822258 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d69c64-b35a-4caa-840f-956fe8eb4dce None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:52.831630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 500/993] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:52.832837 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c34721a-30ac-407b-9918-e5aa61fcf1b2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:52.918617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:52.945222 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c34721a-30ac-407b-9918-e5aa61fcf1b2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:52.979202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f7c9a5-441f-4009-a65b-ca151851ace9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:52.979891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f7c9a5-441f-4009-a65b-ca151851ace9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:52.980171 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f7c9a5-441f-4009-a65b-ca151851ace9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:52.980440 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f7c9a5-441f-4009-a65b-ca151851ace9 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:52.981142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f7c9a5-441f-4009-a65b-ca151851ace9 None admin] RBAC: Adding query filter params (type=image) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:52.981442 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f7c9a5-441f-4009-a65b-ca151851ace9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:52.982809 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f7c9a5-441f-4009-a65b-ca151851ace9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:52.987432 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 494/994] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:52 2022] GET /identity/v3/services?type=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:54.651065 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5dc7fb-a556-41d0-931d-67d0debd5075 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:54.651703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5dc7fb-a556-41d0-931d-67d0debd5075 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:54.651703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5dc7fb-a556-41d0-931d-67d0debd5075 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:54.653175 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 501/995] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:54.658256 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a058dd-6da8-4282-bb88-2b7f891c10c0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:54.658674 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a058dd-6da8-4282-bb88-2b7f891c10c0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:54.658843 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a058dd-6da8-4282-bb88-2b7f891c10c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:54.676212 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-79a058dd-6da8-4282-bb88-2b7f891c10c0 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:54.677673 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-79a058dd-6da8-4282-bb88-2b7f891c10c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:54.746427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 495/996] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:08:54.768956 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5dc7fb-a556-41d0-931d-67d0debd5075 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:54.806793 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:54.849369 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973c4fd-7a03-432f-a119-e45a131abb77 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:54.849692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973c4fd-7a03-432f-a119-e45a131abb77 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:54.849927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973c4fd-7a03-432f-a119-e45a131abb77 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:54.851494 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 496/997] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:08:54.856276 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ea9aa-a912-4b4f-abf5-8ffa44ad145f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:54.856565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ea9aa-a912-4b4f-abf5-8ffa44ad145f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:54.856839 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ea9aa-a912-4b4f-abf5-8ffa44ad145f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:54.874901 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5dc7fb-a556-41d0-931d-67d0debd5075 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:54.876122 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-937ea9aa-a912-4b4f-abf5-8ffa44ad145f None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:54.878632 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-937ea9aa-a912-4b4f-abf5-8ffa44ad145f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:54.914115 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc994ba-6195-4ae0-99c1-a2846b902ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:54.914837 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc994ba-6195-4ae0-99c1-a2846b902ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:54.915126 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc994ba-6195-4ae0-99c1-a2846b902ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:54.915412 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc994ba-6195-4ae0-99c1-a2846b902ce5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:54.916063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc994ba-6195-4ae0-99c1-a2846b902ce5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:54.917419 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc994ba-6195-4ae0-99c1-a2846b902ce5 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:54.929693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-bcc994ba-6195-4ae0-99c1-a2846b902ce5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 14:08:54.929693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:54.929693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:08:54.929693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:08:54.929693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:08:54.929693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:08:54.929693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:08:54.929693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:54.929693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 14:08:54.929693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:54.929693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:08:54.929693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:54.929693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:54.929693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:54.929693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:54.929693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:54.929693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:54.930848 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:54.930848 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:54.930848 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:54.930848 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:54.930848 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:54.930848 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:54.930848 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:08:54.930848 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:08:54.930848 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:08:54.930848 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:08:54.930848 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:54.930848 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:54.930848 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:54.930848 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:54.930848 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:54.930848 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:54.931893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:54.931893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:54.931893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:54.931893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:54.931893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:54.931893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:54.931893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:54.931893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:54.931893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:54.931893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:54.931893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:08:54.931893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:54.931893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 14:08:54.931893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:08:54.931893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 502/998] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:54.936701 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc994ba-6195-4ae0-99c1-a2846b902ce5 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:54.952255 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:54.959511 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 497/999] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:08:54.984149 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ea9aa-a912-4b4f-abf5-8ffa44ad145f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:55.045915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:55.049869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc994ba-6195-4ae0-99c1-a2846b902ce5 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:55.089290 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdcb823-5c7c-467e-a221-3809cc973c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:55.089956 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdcb823-5c7c-467e-a221-3809cc973c3b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:55.090239 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdcb823-5c7c-467e-a221-3809cc973c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:55.090514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdcb823-5c7c-467e-a221-3809cc973c3b None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:55.091240 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdcb823-5c7c-467e-a221-3809cc973c3b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:55.091569 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdcb823-5c7c-467e-a221-3809cc973c3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:55.092924 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdcb823-5c7c-467e-a221-3809cc973c3b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:55.096908 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ea9aa-a912-4b4f-abf5-8ffa44ad145f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:55.099095 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 503/1000] 10.222.0.81 () {56 vars in 1112 bytes} [Tue Jun 7 14:08:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:55.104216 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdcb823-5c7c-467e-a221-3809cc973c3b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:55.138488 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2de41-034a-4fb3-9f6a-f0aa5c5591fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:55.139390 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2de41-034a-4fb3-9f6a-f0aa5c5591fe None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:55.139682 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2de41-034a-4fb3-9f6a-f0aa5c5591fe None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:55.139951 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2de41-034a-4fb3-9f6a-f0aa5c5591fe None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:55.140604 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2de41-034a-4fb3-9f6a-f0aa5c5591fe None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:55.141970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2de41-034a-4fb3-9f6a-f0aa5c5591fe None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:55.154629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-a7e2de41-034a-4fb3-9f6a-f0aa5c5591fe None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:08:55.154629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:55.154629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:08:55.154629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:08:55.154629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:08:55.154629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:08:55.154629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:08:55.154629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:55.154629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:08:55.154629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:55.154629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:08:55.154629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:55.154629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:55.154629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:55.154629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:55.154629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:55.154629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:55.155723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:55.155723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:55.155723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:55.155723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:55.155723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:55.155723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:55.155723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:08:55.155723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:08:55.155723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:08:55.155723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:08:55.155723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:55.155723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:55.155723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:55.155723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:55.155723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:55.155723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:55.162602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:55.162602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:55.162602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:55.162602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:55.162602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:55.162602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:55.162602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:55.162602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:55.162602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:55.162602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:55.162602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:08:55.162602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:55.162602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:08:55.162602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:55.162602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 498/1001] 10.222.0.81 () {56 vars in 1126 bytes} [Tue Jun 7 14:08:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:55.166642 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2de41-034a-4fb3-9f6a-f0aa5c5591fe None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:55.217088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdcb823-5c7c-467e-a221-3809cc973c3b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:55.230839 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:55.255800 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9788f66-3280-4407-b065-7c695201c6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:55.256747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9788f66-3280-4407-b065-7c695201c6aa None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:55.257240 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9788f66-3280-4407-b065-7c695201c6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:55.257688 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9788f66-3280-4407-b065-7c695201c6aa None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:55.258542 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9788f66-3280-4407-b065-7c695201c6aa None admin] RBAC: Adding query filter params (type=compute) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:55.259075 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9788f66-3280-4407-b065-7c695201c6aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:55.260611 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9788f66-3280-4407-b065-7c695201c6aa None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:55.266012 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 504/1002] 10.222.0.81 () {56 vars in 1112 bytes} [Tue Jun 7 14:08:55 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:55.272310 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9788f66-3280-4407-b065-7c695201c6aa None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:55.277822 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2de41-034a-4fb3-9f6a-f0aa5c5591fe None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:55.304647 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:55.314120 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436d1a6-df70-4bc5-b3e5-873d98d3824c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:55.314787 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436d1a6-df70-4bc5-b3e5-873d98d3824c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:55.315070 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436d1a6-df70-4bc5-b3e5-873d98d3824c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:55.315377 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436d1a6-df70-4bc5-b3e5-873d98d3824c None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:55.316090 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436d1a6-df70-4bc5-b3e5-873d98d3824c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:55.316394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436d1a6-df70-4bc5-b3e5-873d98d3824c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:55.317984 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436d1a6-df70-4bc5-b3e5-873d98d3824c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:55.322705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 499/1003] 10.222.0.81 () {56 vars in 1122 bytes} [Tue Jun 7 14:08:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:55.324006 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ff7ba-9d78-44fc-9b2d-0aa1f5b90b66 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:55.324297 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ff7ba-9d78-44fc-9b2d-0aa1f5b90b66 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:55.324571 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ff7ba-9d78-44fc-9b2d-0aa1f5b90b66 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:55.325791 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 500/1004] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:55.328163 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ff7ba-9d78-44fc-9b2d-0aa1f5b90b66 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:55.382192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9788f66-3280-4407-b065-7c695201c6aa None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:55.415889 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fb06b-18e8-4f20-ae10-623b5dd8c8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:55.416543 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fb06b-18e8-4f20-ae10-623b5dd8c8bf None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:55.416879 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fb06b-18e8-4f20-ae10-623b5dd8c8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:55.417102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fb06b-18e8-4f20-ae10-623b5dd8c8bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:55.417886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85fb06b-18e8-4f20-ae10-623b5dd8c8bf None admin] RBAC: Adding query filter params (interface=public, service_id=67f374ee0ac04c9bb4b1ade59699baf6, region_id=RegionOne) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:55.418194 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85fb06b-18e8-4f20-ae10-623b5dd8c8bf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:55.419579 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85fb06b-18e8-4f20-ae10-623b5dd8c8bf None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:55.424784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 505/1005] 10.222.0.81 () {56 vars in 1251 bytes} [Tue Jun 7 14:08:55 2022] GET /identity/v3/endpoints?service_id=67f374ee0ac04c9bb4b1ade59699baf6&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:55.426043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd0892-6f8e-4314-bcd1-def05dd04392 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:55.426337 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd0892-6f8e-4314-bcd1-def05dd04392 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:55.426669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd0892-6f8e-4314-bcd1-def05dd04392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:55.429052 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ff7ba-9d78-44fc-9b2d-0aa1f5b90b66 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:55.443821 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-5bbd0892-6f8e-4314-bcd1-def05dd04392 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:55.445036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5bbd0892-6f8e-4314-bcd1-def05dd04392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:55.458644 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:55.467889 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba80324-df13-4a71-a1dd-cf389ebdfe6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:55.468559 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba80324-df13-4a71-a1dd-cf389ebdfe6b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:55.468843 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba80324-df13-4a71-a1dd-cf389ebdfe6b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:55.469123 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba80324-df13-4a71-a1dd-cf389ebdfe6b None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:55.469833 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba80324-df13-4a71-a1dd-cf389ebdfe6b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:55.470141 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba80324-df13-4a71-a1dd-cf389ebdfe6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:55.471552 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba80324-df13-4a71-a1dd-cf389ebdfe6b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:55.477071 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 501/1006] 10.222.0.81 () {56 vars in 1122 bytes} [Tue Jun 7 14:08:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:55.477840 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba80324-df13-4a71-a1dd-cf389ebdfe6b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:55.515812 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 506/1007] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:08:55.516643 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd0892-6f8e-4314-bcd1-def05dd04392 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:55.539092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:55.582507 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba80324-df13-4a71-a1dd-cf389ebdfe6b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:55.616773 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f681c5e-f653-494a-879e-de64f9c2ba4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:55.617434 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f681c5e-f653-494a-879e-de64f9c2ba4d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:55.617716 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f681c5e-f653-494a-879e-de64f9c2ba4d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:55.617988 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f681c5e-f653-494a-879e-de64f9c2ba4d None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:55.618114 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd0892-6f8e-4314-bcd1-def05dd04392 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:55.618669 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f681c5e-f653-494a-879e-de64f9c2ba4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:55.620331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f681c5e-f653-494a-879e-de64f9c2ba4d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:55.626430 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 502/1008] 10.222.0.81 () {56 vars in 1087 bytes} [Tue Jun 7 14:08:55 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:08:55.627183 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f681c5e-f653-494a-879e-de64f9c2ba4d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:55.663919 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b58e97-161f-464d-86de-7223e6b13239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:55.664634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b58e97-161f-464d-86de-7223e6b13239 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:55.664913 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b58e97-161f-464d-86de-7223e6b13239 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:55.665177 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b58e97-161f-464d-86de-7223e6b13239 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:55.666210 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b58e97-161f-464d-86de-7223e6b13239 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:55.668433 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b58e97-161f-464d-86de-7223e6b13239 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:55.685378 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 507/1009] 10.222.0.81 () {60 vars in 1144 bytes} [Tue Jun 7 14:08:55 2022] POST /identity/v3/endpoints => generated 325 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:08:55.705442 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:55.767919 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f681c5e-f653-494a-879e-de64f9c2ba4d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:55.816374 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fafc000-3b05-480b-a42b-f9a5b2318ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:55.817703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafc000-3b05-480b-a42b-f9a5b2318ff3 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:55.817990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafc000-3b05-480b-a42b-f9a5b2318ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:55.818265 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafc000-3b05-480b-a42b-f9a5b2318ff3 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:55.819146 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fafc000-3b05-480b-a42b-f9a5b2318ff3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:55.820611 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fafc000-3b05-480b-a42b-f9a5b2318ff3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:55.837554 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 503/1010] 10.222.0.81 () {60 vars in 1141 bytes} [Tue Jun 7 14:08:55 2022] POST /identity/v3/services => generated 242 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:08:57.938105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdc22c-c309-418b-aaa8-2b38c29789af None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:57.938636 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdc22c-c309-418b-aaa8-2b38c29789af None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:57.938749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdc22c-c309-418b-aaa8-2b38c29789af None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:57.940235 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 508/1011] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:57.945802 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb835b5-d694-44a2-98b5-ae2f6237485c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:57.946190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb835b5-d694-44a2-98b5-ae2f6237485c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:57.946368 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb835b5-d694-44a2-98b5-ae2f6237485c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:57.963266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4bb835b5-d694-44a2-98b5-ae2f6237485c None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:57.964471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4bb835b5-d694-44a2-98b5-ae2f6237485c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:58.008534 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2505e6f-b1be-463b-a57a-0847eb5ae0d9 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:58.008866 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2505e6f-b1be-463b-a57a-0847eb5ae0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:58.009100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2505e6f-b1be-463b-a57a-0847eb5ae0d9 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:58.010639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 509/1012] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:08:58.015539 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004bdfe-44b8-4dae-82e0-61421d9dcbf4 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:58.015828 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004bdfe-44b8-4dae-82e0-61421d9dcbf4 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:58.016108 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004bdfe-44b8-4dae-82e0-61421d9dcbf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:58.033196 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-b004bdfe-44b8-4dae-82e0-61421d9dcbf4 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:58.034375 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b004bdfe-44b8-4dae-82e0-61421d9dcbf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:58.045466 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 504/1013] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:57 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:08:58.068399 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb835b5-d694-44a2-98b5-ae2f6237485c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:58.111252 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 510/1014] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:58 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:08:58.119191 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:58.131342 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004bdfe-44b8-4dae-82e0-61421d9dcbf4 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:58.143277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:58.182028 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb835b5-d694-44a2-98b5-ae2f6237485c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:58.215872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9c3d0-1998-403e-9dd2-8239c109cf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:58.216661 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9c3d0-1998-403e-9dd2-8239c109cf54 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:58.216941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9c3d0-1998-403e-9dd2-8239c109cf54 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:58.217218 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9c3d0-1998-403e-9dd2-8239c109cf54 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:58.217868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9c3d0-1998-403e-9dd2-8239c109cf54 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:58.219226 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9c3d0-1998-403e-9dd2-8239c109cf54 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:58.231239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-73a9c3d0-1998-403e-9dd2-8239c109cf54 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 14:08:58.231239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:58.231239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:08:58.231239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:08:58.231239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:08:58.231239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:08:58.231239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:08:58.231239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:58.231239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 14:08:58.231239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:58.231239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:08:58.231239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:58.231239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:58.231239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:58.231239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:58.231239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:58.231239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:58.232482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:58.232482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:58.232482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:58.232482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:58.232482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:58.232482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:58.232482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:08:58.232482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:08:58.232482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:08:58.232482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:08:58.232482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:58.232482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:58.232482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:58.232482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:58.232482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:58.232482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:58.233355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:58.233355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:58.233355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:58.233355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:58.233355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:58.233355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:58.233355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:58.233355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:58.233355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:58.233355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:58.233355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:08:58.233355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:58.233355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 14:08:58.233355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:58.233355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 505/1015] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:08:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:08:58.234170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe78c8-f4d1-4f0d-bbd5-d9520e641dd3 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:58.234440 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe78c8-f4d1-4f0d-bbd5-d9520e641dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:58.234627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe78c8-f4d1-4f0d-bbd5-d9520e641dd3 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:58.236043 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 506/1016] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:08:58.238207 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe78c8-f4d1-4f0d-bbd5-d9520e641dd3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:58.245139 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004bdfe-44b8-4dae-82e0-61421d9dcbf4 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:58.279436 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61936bcc-e62b-4819-80da-7fba99bd466c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:58.280618 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61936bcc-e62b-4819-80da-7fba99bd466c None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:58.280897 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61936bcc-e62b-4819-80da-7fba99bd466c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:58.281167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61936bcc-e62b-4819-80da-7fba99bd466c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:58.282667 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61936bcc-e62b-4819-80da-7fba99bd466c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:58.284416 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61936bcc-e62b-4819-80da-7fba99bd466c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:58.286744 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 511/1017] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:58.288062 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ef693-80df-4dae-a83c-0e74f68a218d None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:58.288320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ef693-80df-4dae-a83c-0e74f68a218d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:58.288520 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ef693-80df-4dae-a83c-0e74f68a218d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:58.305923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-840ef693-80df-4dae-a83c-0e74f68a218d None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:08:58.307102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-840ef693-80df-4dae-a83c-0e74f68a218d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:08:58.336799 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:58.340971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe78c8-f4d1-4f0d-bbd5-d9520e641dd3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:58.375283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b527231-52a0-446d-9ab4-1d0e18f05b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:58.375604 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 512/1018] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:08:58 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:08:58.376420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ef693-80df-4dae-a83c-0e74f68a218d None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:58.379054 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b527231-52a0-446d-9ab4-1d0e18f05b07 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:58.380273 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b527231-52a0-446d-9ab4-1d0e18f05b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:58.380555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b527231-52a0-446d-9ab4-1d0e18f05b07 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:58.381274 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b527231-52a0-446d-9ab4-1d0e18f05b07 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:58.381582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b527231-52a0-446d-9ab4-1d0e18f05b07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:58.382951 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b527231-52a0-446d-9ab4-1d0e18f05b07 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:58.386013 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:58.395035 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 507/1019] 10.222.0.81 () {56 vars in 1112 bytes} [Tue Jun 7 14:08:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:58.396124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b527231-52a0-446d-9ab4-1d0e18f05b07 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:58.475036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ef693-80df-4dae-a83c-0e74f68a218d None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:58.495120 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b527231-52a0-446d-9ab4-1d0e18f05b07 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:58.509628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dba597-9640-4f28-8be0-ebbe5f9edfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:58.510288 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dba597-9640-4f28-8be0-ebbe5f9edfab None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:58.510571 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dba597-9640-4f28-8be0-ebbe5f9edfab None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:58.510923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dba597-9640-4f28-8be0-ebbe5f9edfab None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:58.511670 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dba597-9640-4f28-8be0-ebbe5f9edfab None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:58.513094 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dba597-9640-4f28-8be0-ebbe5f9edfab None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:58.529116 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f863135-1bf6-4ad1-8261-9486491fae75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:58.529930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f863135-1bf6-4ad1-8261-9486491fae75 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:58.530329 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f863135-1bf6-4ad1-8261-9486491fae75 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:58.530658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f863135-1bf6-4ad1-8261-9486491fae75 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:58.531404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f863135-1bf6-4ad1-8261-9486491fae75 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:58.531712 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f863135-1bf6-4ad1-8261-9486491fae75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:58.533246 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f863135-1bf6-4ad1-8261-9486491fae75 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:58.539113 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 508/1020] 10.222.0.81 () {56 vars in 1112 bytes} [Tue Jun 7 14:08:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:58.539565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 513/1021] 10.222.0.81 () {60 vars in 1141 bytes} [Tue Jun 7 14:08:58 2022] POST /identity/v3/projects => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:08:58.539731 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f863135-1bf6-4ad1-8261-9486491fae75 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:58.549019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dba597-9640-4f28-8be0-ebbe5f9edfab None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:58.572662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:58.631752 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:58.660641 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f863135-1bf6-4ad1-8261-9486491fae75 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:58.684074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dba597-9640-4f28-8be0-ebbe5f9edfab None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:58.700718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba9444-791a-411c-9ae5-8d6b75480871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:58.701532 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba9444-791a-411c-9ae5-8d6b75480871 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:58.701820 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba9444-791a-411c-9ae5-8d6b75480871 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:58.702099 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba9444-791a-411c-9ae5-8d6b75480871 None admin] PATH_INFO: `/v3/services/image` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:58.702942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ba9444-791a-411c-9ae5-8d6b75480871 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:58.704499 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ba9444-791a-411c-9ae5-8d6b75480871 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:58.718367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-69ba9444-791a-411c-9ae5-8d6b75480871 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 14:08:58.718367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:58.718367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:08:58.718367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:08:58.718367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:08:58.718367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:08:58.718367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:08:58.718367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:58.718367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 14:08:58.718367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:58.718367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:08:58.718367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:58.718367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:08:58.718367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:08:58.718367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:08:58.718367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:08:58.718367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:08:58.719534 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:08:58.719534 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:08:58.719534 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:08:58.719534 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:08:58.719534 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:08:58.719534 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:08:58.719534 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:08:58.719534 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:08:58.719534 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:08:58.719534 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:08:58.719534 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:08:58.719534 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:08:58.719534 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:08:58.719534 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:08:58.719534 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:08:58.719534 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:08:58.720404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:08:58.720404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:08:58.720404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:08:58.720404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:08:58.720404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:08:58.720404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:08:58.720404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:08:58.720404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:08:58.720404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:08:58.720404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:08:58.720404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:08:58.720404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:08:58.720404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 14:08:58.720404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:08:58.720404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 509/1022] 10.222.0.81 () {56 vars in 1105 bytes} [Tue Jun 7 14:08:58 2022] GET /identity/v3/services/image => generated 86 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:08:58.725453 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad61aa-b871-4499-839e-9eefc648066c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:58.726081 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad61aa-b871-4499-839e-9eefc648066c None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:58.726356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad61aa-b871-4499-839e-9eefc648066c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:58.726632 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad61aa-b871-4499-839e-9eefc648066c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:58.727043 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba9444-791a-411c-9ae5-8d6b75480871 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:58.727493 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad61aa-b871-4499-839e-9eefc648066c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:58.728905 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad61aa-b871-4499-839e-9eefc648066c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:58.751933 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 514/1023] 10.222.0.81 () {60 vars in 1144 bytes} [Tue Jun 7 14:08:58 2022] POST /identity/v3/endpoints => generated 334 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:08:58.838330 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:58.850461 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba9444-791a-411c-9ae5-8d6b75480871 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:58.905897 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e174f8d-e8a4-4ddb-87a5-068cdee13d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:58.906977 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e174f8d-e8a4-4ddb-87a5-068cdee13d9b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:58.907502 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e174f8d-e8a4-4ddb-87a5-068cdee13d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:58.907985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e174f8d-e8a4-4ddb-87a5-068cdee13d9b None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:58.908946 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e174f8d-e8a4-4ddb-87a5-068cdee13d9b None admin] RBAC: Adding query filter params (name=image) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:58.909462 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e174f8d-e8a4-4ddb-87a5-068cdee13d9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:58.911587 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e174f8d-e8a4-4ddb-87a5-068cdee13d9b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:58.925859 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 510/1024] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:08:58.932568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad61aa-b871-4499-839e-9eefc648066c None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:59.054327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad61aa-b871-4499-839e-9eefc648066c None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:59.073332 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:08:59.093871 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc131e91-51a5-4602-bc11-b1fb8f4aaac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:59.094540 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc131e91-51a5-4602-bc11-b1fb8f4aaac3 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:59.094822 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc131e91-51a5-4602-bc11-b1fb8f4aaac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:59.095095 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc131e91-51a5-4602-bc11-b1fb8f4aaac3 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:59.095853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc131e91-51a5-4602-bc11-b1fb8f4aaac3 None admin] RBAC: Adding query filter params (type=image) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:59.096165 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc131e91-51a5-4602-bc11-b1fb8f4aaac3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:59.097511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc131e91-51a5-4602-bc11-b1fb8f4aaac3 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:59.102785 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 515/1025] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:08:58 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:59.108408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e174f8d-e8a4-4ddb-87a5-068cdee13d9b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:59.224653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e174f8d-e8a4-4ddb-87a5-068cdee13d9b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:59.259555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d849e-25ff-4a54-bcd8-cb40e9eeec7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:59.260225 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d849e-25ff-4a54-bcd8-cb40e9eeec7d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:59.260512 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d849e-25ff-4a54-bcd8-cb40e9eeec7d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:59.260793 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d849e-25ff-4a54-bcd8-cb40e9eeec7d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:59.261593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d849e-25ff-4a54-bcd8-cb40e9eeec7d None admin] RBAC: Adding query filter params (interface=public, service_id=1475f50db24f409da5b917f16e548e3d, region_id=RegionOne) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:08:59.261913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d849e-25ff-4a54-bcd8-cb40e9eeec7d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:59.263302 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d849e-25ff-4a54-bcd8-cb40e9eeec7d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:59.273538 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 511/1026] 10.222.0.81 () {56 vars in 1251 bytes} [Tue Jun 7 14:08:59 2022] GET /identity/v3/endpoints?service_id=1475f50db24f409da5b917f16e548e3d&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:08:59.275129 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc131e91-51a5-4602-bc11-b1fb8f4aaac3 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:08:59.377630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc131e91-51a5-4602-bc11-b1fb8f4aaac3 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:08:59.413569 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caeefc2-91c1-482a-bdf8-6d3015e2b900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:08:59.414255 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caeefc2-91c1-482a-bdf8-6d3015e2b900 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:08:59.414523 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caeefc2-91c1-482a-bdf8-6d3015e2b900 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:08:59.414803 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caeefc2-91c1-482a-bdf8-6d3015e2b900 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:08:59.415518 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caeefc2-91c1-482a-bdf8-6d3015e2b900 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:08:59.416905 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caeefc2-91c1-482a-bdf8-6d3015e2b900 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:08:59.427271 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 516/1027] 10.222.0.81 () {56 vars in 1087 bytes} [Tue Jun 7 14:08:59 2022] GET /identity/v3/services => generated 1957 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:09:00.908487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8489a-e68e-4e7c-bd2b-be1f0d36ed6a None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:00.909014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8489a-e68e-4e7c-bd2b-be1f0d36ed6a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:00.909092 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8489a-e68e-4e7c-bd2b-be1f0d36ed6a None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:00.910511 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 512/1028] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:09:00.916150 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a3dc3-40c3-4ea7-aac6-407e01de35bd None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:00.916605 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a3dc3-40c3-4ea7-aac6-407e01de35bd None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:00.916797 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a3dc3-40c3-4ea7-aac6-407e01de35bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:00.934606 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-140a3dc3-40c3-4ea7-aac6-407e01de35bd None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:09:00.935869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-140a3dc3-40c3-4ea7-aac6-407e01de35bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:09:00.978769 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:01.007888 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 517/1029] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:09:00 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:09:01.032281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8489a-e68e-4e7c-bd2b-be1f0d36ed6a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:01.052156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:01.140023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8489a-e68e-4e7c-bd2b-be1f0d36ed6a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:01.146846 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd3042-add4-4c67-a6c6-6a8e98b351c8 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:01.147132 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd3042-add4-4c67-a6c6-6a8e98b351c8 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:01.147442 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd3042-add4-4c67-a6c6-6a8e98b351c8 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:01.148834 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 518/1030] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:09:01.154335 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ace445-1d6e-4a7b-b6c6-9f46dc2f678f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:01.154625 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ace445-1d6e-4a7b-b6c6-9f46dc2f678f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:01.154911 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ace445-1d6e-4a7b-b6c6-9f46dc2f678f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:01.173302 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-f3ace445-1d6e-4a7b-b6c6-9f46dc2f678f None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:09:01.174488 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f3ace445-1d6e-4a7b-b6c6-9f46dc2f678f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:09:01.179491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff49f7-c0bd-4472-8677-054dcecc61bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:01.180295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff49f7-c0bd-4472-8677-054dcecc61bc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:01.180579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff49f7-c0bd-4472-8677-054dcecc61bc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:01.180854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff49f7-c0bd-4472-8677-054dcecc61bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:01.182060 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff49f7-c0bd-4472-8677-054dcecc61bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:01.183520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff49f7-c0bd-4472-8677-054dcecc61bc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:01.185166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 513/1031] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:09:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:09:01.190830 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff49f7-c0bd-4472-8677-054dcecc61bc None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:01.252183 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 519/1032] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:09:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:09:01.277358 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ace445-1d6e-4a7b-b6c6-9f46dc2f678f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:01.282851 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:01.302403 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff49f7-c0bd-4472-8677-054dcecc61bc None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:01.339868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cac9ee-a9f6-4da8-b73c-bca448e969e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:01.340655 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cac9ee-a9f6-4da8-b73c-bca448e969e3 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:01.340965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cac9ee-a9f6-4da8-b73c-bca448e969e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:01.341316 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cac9ee-a9f6-4da8-b73c-bca448e969e3 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:01.342035 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cac9ee-a9f6-4da8-b73c-bca448e969e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:01.343723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cac9ee-a9f6-4da8-b73c-bca448e969e3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:01.370253 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 514/1033] 10.222.0.81 () {60 vars in 1132 bytes} [Tue Jun 7 14:09:01 2022] POST /identity/v3/users => generated 285 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:09:01.394575 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ace445-1d6e-4a7b-b6c6-9f46dc2f678f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:01.398537 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:01.436217 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d70d17-57be-48e9-a3d6-101ba8decf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:01.436965 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d70d17-57be-48e9-a3d6-101ba8decf34 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:01.437251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d70d17-57be-48e9-a3d6-101ba8decf34 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:01.437526 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d70d17-57be-48e9-a3d6-101ba8decf34 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:01.450382 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d70d17-57be-48e9-a3d6-101ba8decf34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:09:01.458925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-86d70d17-57be-48e9-a3d6-101ba8decf34 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 14:09:01.458925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:01.458925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:01.458925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:01.458925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:01.458925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:01.458925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:01.458925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:01.458925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:01.458925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:01.458925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:01.458925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:01.458925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:09:01.458925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:09:01.458925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:09:01.458925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:09:01.460073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:09:01.460073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:09:01.460073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:09:01.460073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:09:01.460073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:01.460073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:01.460073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:01.460073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:01.460073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:01.460073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:01.460073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:01.460073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:01.460073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:01.460073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:01.460073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:01.460073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:01.466669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:01.466669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:01.466669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:01.466669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:01.466669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:09:01.466669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:09:01.466669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:09:01.466669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:09:01.466669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:09:01.466669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:09:01.466669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 14:09:01.466669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:09:01.466669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 520/1034] 10.222.0.81 () {56 vars in 1120 bytes} [Tue Jun 7 14:09:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:09:01.469124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cac9ee-a9f6-4da8-b73c-bca448e969e3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:01.581871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cac9ee-a9f6-4da8-b73c-bca448e969e3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:01.620026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852a6ec-1b53-4673-b64b-d0e282bee9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:01.620955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852a6ec-1b53-4673-b64b-d0e282bee9bb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:01.621236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852a6ec-1b53-4673-b64b-d0e282bee9bb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:01.621511 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852a6ec-1b53-4673-b64b-d0e282bee9bb None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:01.622268 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852a6ec-1b53-4673-b64b-d0e282bee9bb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:01.622572 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852a6ec-1b53-4673-b64b-d0e282bee9bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:01.623989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852a6ec-1b53-4673-b64b-d0e282bee9bb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:01.632683 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 515/1035] 10.222.0.81 () {56 vars in 1115 bytes} [Tue Jun 7 14:09:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:01.640297 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d70d17-57be-48e9-a3d6-101ba8decf34 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:01.700373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d1598-562e-49d0-8e04-8f3abcaa838b None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:01.700674 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d1598-562e-49d0-8e04-8f3abcaa838b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:01.700858 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d1598-562e-49d0-8e04-8f3abcaa838b None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:01.702241 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 516/1036] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:09:01.707127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd8044-ac08-488a-91b3-48efd6845538 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:01.707478 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd8044-ac08-488a-91b3-48efd6845538 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:01.707777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd8044-ac08-488a-91b3-48efd6845538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:01.713441 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:01.730310 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-aebd8044-ac08-488a-91b3-48efd6845538 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:09:01.731555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-aebd8044-ac08-488a-91b3-48efd6845538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:09:01.757399 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d70d17-57be-48e9-a3d6-101ba8decf34 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:01.793168 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bbe09-0616-4f21-83db-d43815554ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:01.794007 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bbe09-0616-4f21-83db-d43815554ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:01.794291 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bbe09-0616-4f21-83db-d43815554ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:01.794568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bbe09-0616-4f21-83db-d43815554ed3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:01.805619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 517/1037] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:09:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:09:01.808257 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212bbe09-0616-4f21-83db-d43815554ed3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:01.809734 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212bbe09-0616-4f21-83db-d43815554ed3 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:01.828160 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-212bbe09-0616-4f21-83db-d43815554ed3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:09:01.828160 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:01.828160 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:01.828160 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:01.828160 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:01.828160 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:01.828160 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:01.828160 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:01.828160 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:01.828160 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:01.828160 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:01.828160 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:01.828160 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:09:01.828160 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:09:01.828160 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:09:01.829140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:09:01.829140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:01.829140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:01.829140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:01.829140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:01.829140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:01.829140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:01.829140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:01.829140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:01.829140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:01.829140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:01.829140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:01.829140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:01.829140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:01.829140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:01.829140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:01.829140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:01.830082 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:09:01.830082 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:09:01.830082 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:09:01.830082 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:09:01.830325 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd8044-ac08-488a-91b3-48efd6845538 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:01.834240 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 521/1038] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:09:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:09:01.835306 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bbe09-0616-4f21-83db-d43815554ed3 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:01.856371 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:01.938452 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd8044-ac08-488a-91b3-48efd6845538 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:01.950773 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bbe09-0616-4f21-83db-d43815554ed3 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:01.969615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:01.976180 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cb1eab-17e4-4b6b-b660-6b118514d10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:01.976975 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb1eab-17e4-4b6b-b660-6b118514d10f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:01.977263 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb1eab-17e4-4b6b-b660-6b118514d10f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:01.977540 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb1eab-17e4-4b6b-b660-6b118514d10f None admin] PATH_INFO: `/v3/services/image` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:01.978194 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cb1eab-17e4-4b6b-b660-6b118514d10f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:01.979599 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cb1eab-17e4-4b6b-b660-6b118514d10f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:01.988449 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50160d04-96e8-45b1-8183-e46aef8e6c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:01.989745 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50160d04-96e8-45b1-8183-e46aef8e6c85 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:01.990026 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50160d04-96e8-45b1-8183-e46aef8e6c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:01.990300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50160d04-96e8-45b1-8183-e46aef8e6c85 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:01.991028 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50160d04-96e8-45b1-8183-e46aef8e6c85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:01.991383 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50160d04-96e8-45b1-8183-e46aef8e6c85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:01.992094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-89cb1eab-17e4-4b6b-b660-6b118514d10f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 14:09:01.992094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:01.992094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:09:01.992094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:09:01.992094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:09:01.992094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:09:01.992094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:09:01.992094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:09:01.992094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 14:09:01.992094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:09:01.992094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:09:01.992094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:09:01.992094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:01.992094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:01.992094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:01.992094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:01.992094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:01.993057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:01.993057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:01.993057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:01.993057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:01.993057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:01.993057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:01.993057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:09:01.993057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:09:01.993057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:09:01.993057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:09:01.993057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:01.993057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:01.993057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:01.993057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:01.993057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:01.993057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:01.994058 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:01.994058 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:01.994058 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:01.994058 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:01.994058 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:01.994058 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:01.994058 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:01.994058 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:01.994058 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:01.994058 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:01.994058 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:09:01.994058 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:09:01.994058 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 14:09:01.994058 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:09:01.995118 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50160d04-96e8-45b1-8183-e46aef8e6c85 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:01.995190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 518/1039] 10.222.0.81 () {56 vars in 1105 bytes} [Tue Jun 7 14:09:01 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:09:01.999413 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cb1eab-17e4-4b6b-b660-6b118514d10f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:02.009745 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 522/1040] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:09:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:09:02.015249 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50160d04-96e8-45b1-8183-e46aef8e6c85 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:02.097892 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:02.113090 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cb1eab-17e4-4b6b-b660-6b118514d10f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:02.122526 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50160d04-96e8-45b1-8183-e46aef8e6c85 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:02.150743 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba99da-5397-4295-bce7-d13b67d22952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:02.151737 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba99da-5397-4295-bce7-d13b67d22952 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:02.152203 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba99da-5397-4295-bce7-d13b67d22952 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:02.152653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba99da-5397-4295-bce7-d13b67d22952 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:02.153568 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba99da-5397-4295-bce7-d13b67d22952 None admin] RBAC: Adding query filter params (name=image) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:02.154062 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba99da-5397-4295-bce7-d13b67d22952 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:02.155648 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba99da-5397-4295-bce7-d13b67d22952 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:02.161110 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 519/1041] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:09:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:09:02.167157 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba99da-5397-4295-bce7-d13b67d22952 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:02.167828 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc99a43-f463-4c05-8707-662a86cb79a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:02.168640 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc99a43-f463-4c05-8707-662a86cb79a4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:02.168928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc99a43-f463-4c05-8707-662a86cb79a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:02.169205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc99a43-f463-4c05-8707-662a86cb79a4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:02.177524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc99a43-f463-4c05-8707-662a86cb79a4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:02.178982 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc99a43-f463-4c05-8707-662a86cb79a4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:02.194524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-7bc99a43-f463-4c05-8707-662a86cb79a4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:09:02.194524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:02.194524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:02.194524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:02.194524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:02.194524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:02.194524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:02.194524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:02.194524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:02.194524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:02.194524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:02.194524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:02.194524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:09:02.194524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:09:02.194524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:09:02.194524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:09:02.195639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:02.195639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:02.195639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:09:02.195639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:09:02.195639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:09:02.195639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:09:02.195639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:02.195639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:02.195639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:02.195639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:02.195639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:02.195639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:02.195639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:02.195639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:02.195639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:02.195639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:02.195639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:02.195639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:02.196614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:02.196614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:02.196614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:09:02.196614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:09:02.196614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:09:02.196614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:09:02.196614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:09:02.196614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:09:02.196614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:09:02.196614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:09:02.202318 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 523/1042] 10.222.0.81 () {56 vars in 1093 bytes} [Tue Jun 7 14:09:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:09:02.203463 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc99a43-f463-4c05-8707-662a86cb79a4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:02.232641 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:02.282699 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba99da-5397-4295-bce7-d13b67d22952 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:02.311233 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc99a43-f463-4c05-8707-662a86cb79a4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:02.320198 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6671b6-cab9-4c0d-ab3a-1c00ffe87567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:02.321111 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6671b6-cab9-4c0d-ab3a-1c00ffe87567 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:02.321588 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6671b6-cab9-4c0d-ab3a-1c00ffe87567 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:02.322055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6671b6-cab9-4c0d-ab3a-1c00ffe87567 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:02.322761 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6671b6-cab9-4c0d-ab3a-1c00ffe87567 None admin] RBAC: Adding query filter params (type=image) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:02.323069 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6671b6-cab9-4c0d-ab3a-1c00ffe87567 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:02.324453 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6671b6-cab9-4c0d-ab3a-1c00ffe87567 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:02.334311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 520/1043] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:09:02 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:02.335599 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6671b6-cab9-4c0d-ab3a-1c00ffe87567 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:02.347295 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6a40a-52db-4aac-985a-b3cbfe60c400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:02.347992 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6a40a-52db-4aac-985a-b3cbfe60c400 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:02.348832 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6a40a-52db-4aac-985a-b3cbfe60c400 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:02.349068 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6a40a-52db-4aac-985a-b3cbfe60c400 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:02.349906 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6a40a-52db-4aac-985a-b3cbfe60c400 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:02.350214 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6a40a-52db-4aac-985a-b3cbfe60c400 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:02.351678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6a40a-52db-4aac-985a-b3cbfe60c400 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:02.354465 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:02.370669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 524/1044] 10.222.0.81 () {56 vars in 1133 bytes} [Tue Jun 7 14:09:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:09:02.376119 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6a40a-52db-4aac-985a-b3cbfe60c400 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:02.443639 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6671b6-cab9-4c0d-ab3a-1c00ffe87567 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:02.468349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:02.481885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763c386-556f-45e8-9a73-abbde02538b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:02.482548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763c386-556f-45e8-9a73-abbde02538b4 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:02.482849 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763c386-556f-45e8-9a73-abbde02538b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:02.483104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763c386-556f-45e8-9a73-abbde02538b4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:02.483876 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5763c386-556f-45e8-9a73-abbde02538b4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:02.485236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5763c386-556f-45e8-9a73-abbde02538b4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:02.486377 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6a40a-52db-4aac-985a-b3cbfe60c400 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:02.505724 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 521/1045] 10.222.0.81 () {60 vars in 1144 bytes} [Tue Jun 7 14:09:02 2022] POST /identity/v3/endpoints => generated 327 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:09:02.544572 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff94f06-fafa-4378-9e2c-13db882c7772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:02.545240 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff94f06-fafa-4378-9e2c-13db882c7772 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:02.545524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff94f06-fafa-4378-9e2c-13db882c7772 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:02.545798 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff94f06-fafa-4378-9e2c-13db882c7772 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:02.614888 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:02.628338 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff94f06-fafa-4378-9e2c-13db882c7772 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:02.630420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff94f06-fafa-4378-9e2c-13db882c7772 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:02.742442 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 525/1046] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:09:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 07 14:09:02.748553 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763c386-556f-45e8-9a73-abbde02538b4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:02.851831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763c386-556f-45e8-9a73-abbde02538b4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:02.885428 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a660bc-024a-47e4-8051-d09da6fd5609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:02.886219 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a660bc-024a-47e4-8051-d09da6fd5609 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:02.886497 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a660bc-024a-47e4-8051-d09da6fd5609 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:02.886765 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a660bc-024a-47e4-8051-d09da6fd5609 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:02.898322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a660bc-024a-47e4-8051-d09da6fd5609 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:02.899968 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a660bc-024a-47e4-8051-d09da6fd5609 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:02.917993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-24a660bc-024a-47e4-8051-d09da6fd5609 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:09:02.917993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:02.917993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:02.917993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:02.917993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:02.917993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:02.917993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:02.917993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:02.917993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:02.917993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:02.917993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:02.917993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:02.917993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:09:02.917993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:09:02.917993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:09:02.918960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:09:02.918960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:02.918960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:02.918960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:02.918960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:02.918960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:02.918960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:02.918960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:02.918960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:02.918960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:02.918960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:02.918960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:02.918960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:02.918960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:02.918960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:02.918960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:02.918960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:02.920229 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:09:02.920229 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:09:02.920229 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:09:02.920229 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:09:02.923540 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 522/1047] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:09:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:09:02.924908 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff94f06-fafa-4378-9e2c-13db882c7772 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:03.018220 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:03.029805 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff94f06-fafa-4378-9e2c-13db882c7772 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:03.065034 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d74da-2555-4be3-b0c4-dd92da620913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:03.065703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d74da-2555-4be3-b0c4-dd92da620913 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:03.065983 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d74da-2555-4be3-b0c4-dd92da620913 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:03.066257 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d74da-2555-4be3-b0c4-dd92da620913 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:03.067088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d74da-2555-4be3-b0c4-dd92da620913 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:03.069155 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d74da-2555-4be3-b0c4-dd92da620913 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:03.069155 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d74da-2555-4be3-b0c4-dd92da620913 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:03.085252 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 526/1048] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:03.086782 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a660bc-024a-47e4-8051-d09da6fd5609 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:03.124614 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:03.205155 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a660bc-024a-47e4-8051-d09da6fd5609 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:03.240379 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe152a-0581-4672-810d-59fb17519b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:03.241181 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe152a-0581-4672-810d-59fb17519b98 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:03.241457 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe152a-0581-4672-810d-59fb17519b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:03.241726 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe152a-0581-4672-810d-59fb17519b98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:03.249325 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe152a-0581-4672-810d-59fb17519b98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:03.250769 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe152a-0581-4672-810d-59fb17519b98 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:03.265884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-78fe152a-0581-4672-810d-59fb17519b98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:09:03.265884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:03.265884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:03.265884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:03.265884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:03.265884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:03.265884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:03.265884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:03.265884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:03.265884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:03.265884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:03.265884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:03.265884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:09:03.265884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:09:03.265884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:09:03.266921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:09:03.266921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:03.266921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:03.266921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:03.266921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:03.266921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:03.266921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:03.266921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:03.266921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:03.266921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:03.266921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:03.266921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:03.266921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:03.266921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:03.266921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:03.266921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:03.266921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:03.268349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:09:03.268349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:09:03.268349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:09:03.268349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:09:03.268349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:09:03.268349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:09:03.268349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:09:03.268349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:09:03.271289 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 523/1049] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:09:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:09:03.273406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d74da-2555-4be3-b0c4-dd92da620913 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:03.375693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d74da-2555-4be3-b0c4-dd92da620913 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:03.413570 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0b8d9-6e00-469b-b6b1-69615b50da28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:03.414259 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0b8d9-6e00-469b-b6b1-69615b50da28 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:03.414546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0b8d9-6e00-469b-b6b1-69615b50da28 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:03.414825 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0b8d9-6e00-469b-b6b1-69615b50da28 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:03.415664 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0b8d9-6e00-469b-b6b1-69615b50da28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:03.415977 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0b8d9-6e00-469b-b6b1-69615b50da28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:03.417439 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0b8d9-6e00-469b-b6b1-69615b50da28 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:03.431270 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 527/1050] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:09:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:03.437141 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe152a-0581-4672-810d-59fb17519b98 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:03.529166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:03.539694 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe152a-0581-4672-810d-59fb17519b98 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:03.573751 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc8779-2a9b-413a-a40f-92fee4f9e7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:03.574491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc8779-2a9b-413a-a40f-92fee4f9e7bd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:03.574848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc8779-2a9b-413a-a40f-92fee4f9e7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:03.575189 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc8779-2a9b-413a-a40f-92fee4f9e7bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:03.576102 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc8779-2a9b-413a-a40f-92fee4f9e7bd None admin] RBAC: Adding query filter params (role.id=855dcf6914224d5c907aa41ec8c2e0c4, scope.project.id=f67bad575f8b4a47bee7c314497cb2fe, user.id=2ae726fbc9cf40e68c7587552a118a7a) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:03.576484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc8779-2a9b-413a-a40f-92fee4f9e7bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:03.577973 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc8779-2a9b-413a-a40f-92fee4f9e7bd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:03.591297 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 524/1051] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:09:03 2022] GET /identity/v3/role_assignments?user.id=2ae726fbc9cf40e68c7587552a118a7a&scope.project.id=f67bad575f8b4a47bee7c314497cb2fe&role.id=855dcf6914224d5c907aa41ec8c2e0c4 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:03.739038 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c78e2-c19e-4f94-821c-86db34b00a75 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:03.739445 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c78e2-c19e-4f94-821c-86db34b00a75 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:03.739886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c78e2-c19e-4f94-821c-86db34b00a75 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:03.741508 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 528/1052] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:09:03.747438 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa517234-bbb3-403e-a054-1a2057b0d2c5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:03.747627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa517234-bbb3-403e-a054-1a2057b0d2c5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:03.747910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa517234-bbb3-403e-a054-1a2057b0d2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:03.766201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-fa517234-bbb3-403e-a054-1a2057b0d2c5 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:09:03.767460 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fa517234-bbb3-403e-a054-1a2057b0d2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:09:03.847375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 525/1053] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:09:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:09:03.865613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203c78e2-c19e-4f94-821c-86db34b00a75 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:03.878723 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:03.971390 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203c78e2-c19e-4f94-821c-86db34b00a75 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:04.006373 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4970e1-f52d-457a-8f9c-6a2366ed6b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:04.007155 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4970e1-f52d-457a-8f9c-6a2366ed6b6e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:04.007466 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4970e1-f52d-457a-8f9c-6a2366ed6b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:04.007740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4970e1-f52d-457a-8f9c-6a2366ed6b6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:04.017879 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4970e1-f52d-457a-8f9c-6a2366ed6b6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:09:04.026286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-0a4970e1-f52d-457a-8f9c-6a2366ed6b6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:09:04.026286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:04.026286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:04.026286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:04.026286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:04.026286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:04.026286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:04.026286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:04.026286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:04.026286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:04.026286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:04.026286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:04.026286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:09:04.026286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:09:04.026286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:09:04.026286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:09:04.027917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:09:04.027917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:09:04.027917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:09:04.027917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:09:04.027917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:04.027917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:04.027917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:04.027917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:04.027917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:04.027917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:04.027917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:04.027917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:04.027917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:04.027917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:04.027917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:04.027917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:04.028813 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:04.028813 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:04.028813 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:04.028813 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:04.028813 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:09:04.028813 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:09:04.028813 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:09:04.028813 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:09:04.028813 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:09:04.028813 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:09:04.028813 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:09:04.028813 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:09:04.031378 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 529/1054] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:09:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:09:04.033521 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa517234-bbb3-403e-a054-1a2057b0d2c5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:04.122960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:04.138422 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa517234-bbb3-403e-a054-1a2057b0d2c5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:04.178801 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5cda5-cb7d-42da-b185-b1617619f9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:04.179510 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5cda5-cb7d-42da-b185-b1617619f9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:04.179798 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5cda5-cb7d-42da-b185-b1617619f9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:04.180076 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5cda5-cb7d-42da-b185-b1617619f9f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:04.180830 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5cda5-cb7d-42da-b185-b1617619f9f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:04.181138 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5cda5-cb7d-42da-b185-b1617619f9f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:04.182502 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5cda5-cb7d-42da-b185-b1617619f9f4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:04.195868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 526/1055] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:09:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:04.196911 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4970e1-f52d-457a-8f9c-6a2366ed6b6e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:04.271415 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:04.304840 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4970e1-f52d-457a-8f9c-6a2366ed6b6e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:04.340390 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21225ca0-1db8-433e-bdf4-548f788434b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:04.341262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21225ca0-1db8-433e-bdf4-548f788434b4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:04.341546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21225ca0-1db8-433e-bdf4-548f788434b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:04.341822 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21225ca0-1db8-433e-bdf4-548f788434b4 None admin] PATH_INFO: `/v3/users/0bf1d2d1c3344504a1eef850b1a420b4` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:04.356549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21225ca0-1db8-433e-bdf4-548f788434b4 None admin] RBAC: Authorizing `identity:get_user(user_id=0bf1d2d1c3344504a1eef850b1a420b4)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:04.358005 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21225ca0-1db8-433e-bdf4-548f788434b4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:04.364907 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 530/1056] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:09:04 2022] GET /identity/v3/users/0bf1d2d1c3344504a1eef850b1a420b4 => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:04.365979 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5cda5-cb7d-42da-b185-b1617619f9f4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:04.467748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5cda5-cb7d-42da-b185-b1617619f9f4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:04.500870 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2407953-a04c-4e19-954a-319d0b17379d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:04.501528 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2407953-a04c-4e19-954a-319d0b17379d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:04.501813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2407953-a04c-4e19-954a-319d0b17379d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:04.502088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2407953-a04c-4e19-954a-319d0b17379d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:04.523554 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:04.576136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2407953-a04c-4e19-954a-319d0b17379d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:04.577587 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2407953-a04c-4e19-954a-319d0b17379d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:04.679255 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 527/1057] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:09:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:09:04.682130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21225ca0-1db8-433e-bdf4-548f788434b4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:04.784419 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21225ca0-1db8-433e-bdf4-548f788434b4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:04.811135 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:04.820891 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f98ec-a86a-463a-94f2-da2c59fe3f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:04.821765 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f98ec-a86a-463a-94f2-da2c59fe3f8f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:04.822049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f98ec-a86a-463a-94f2-da2c59fe3f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:04.822324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f98ec-a86a-463a-94f2-da2c59fe3f8f None admin] PATH_INFO: `/v3/projects/288242db145547d480bf1522f1d0440f` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:04.823555 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f98ec-a86a-463a-94f2-da2c59fe3f8f None admin] RBAC: Authorizing `identity:get_project(project_id=288242db145547d480bf1522f1d0440f)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:04.824987 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f98ec-a86a-463a-94f2-da2c59fe3f8f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:04.831839 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 531/1058] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:09:04 2022] GET /identity/v3/projects/288242db145547d480bf1522f1d0440f => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:04.832866 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2407953-a04c-4e19-954a-319d0b17379d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:04.855073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9a962-f056-4e3c-9f47-b60bc7893035 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:04.855402 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9a962-f056-4e3c-9f47-b60bc7893035 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:04.855654 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9a962-f056-4e3c-9f47-b60bc7893035 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:04.857231 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 532/1059] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:09:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:09:04.861935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8378a-200b-4d9c-9b18-c08bd8f56e63 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:04.862226 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8378a-200b-4d9c-9b18-c08bd8f56e63 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:04.862507 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8378a-200b-4d9c-9b18-c08bd8f56e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:04.881671 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-5af8378a-200b-4d9c-9b18-c08bd8f56e63 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:09:04.883191 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5af8378a-200b-4d9c-9b18-c08bd8f56e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:09:04.888483 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:04.950997 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2407953-a04c-4e19-954a-319d0b17379d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:04.965262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 533/1060] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:09:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:09:04.986648 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01d620-dba8-41de-ab65-73a02e4d4e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:04.987247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8378a-200b-4d9c-9b18-c08bd8f56e63 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:04.987359 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01d620-dba8-41de-ab65-73a02e4d4e80 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:04.987610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01d620-dba8-41de-ab65-73a02e4d4e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:04.987883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01d620-dba8-41de-ab65-73a02e4d4e80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:04.989277 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01d620-dba8-41de-ab65-73a02e4d4e80 None admin] RBAC: Adding query filter params (role.id=82bbe3651b9b4f378fd150d93357b2b1, scope.project.id=288242db145547d480bf1522f1d0440f, user.id=0bf1d2d1c3344504a1eef850b1a420b4) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:04.989828 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01d620-dba8-41de-ab65-73a02e4d4e80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:04.991292 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01d620-dba8-41de-ab65-73a02e4d4e80 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:05.007246 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 528/1061] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:09:04 2022] GET /identity/v3/role_assignments?user.id=0bf1d2d1c3344504a1eef850b1a420b4&scope.project.id=288242db145547d480bf1522f1d0440f&role.id=82bbe3651b9b4f378fd150d93357b2b1 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:09:05.109687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8378a-200b-4d9c-9b18-c08bd8f56e63 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:05.138558 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:05.149563 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603e4af-0125-4240-baf4-133ff216f84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:05.150153 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603e4af-0125-4240-baf4-133ff216f84a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:05.150435 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603e4af-0125-4240-baf4-133ff216f84a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:05.150711 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603e4af-0125-4240-baf4-133ff216f84a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:05.229275 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603e4af-0125-4240-baf4-133ff216f84a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:05.230746 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603e4af-0125-4240-baf4-133ff216f84a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:05.330174 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 534/1062] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:09:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 07 14:09:05.336040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01d620-dba8-41de-ab65-73a02e4d4e80 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:05.438433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01d620-dba8-41de-ab65-73a02e4d4e80 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:05.452224 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:05.475651 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33076ecd-b347-4e00-b046-bb2f4a2f7588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:05.476715 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33076ecd-b347-4e00-b046-bb2f4a2f7588 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:05.477144 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33076ecd-b347-4e00-b046-bb2f4a2f7588 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:05.477580 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33076ecd-b347-4e00-b046-bb2f4a2f7588 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:05.488245 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33076ecd-b347-4e00-b046-bb2f4a2f7588 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:05.490119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33076ecd-b347-4e00-b046-bb2f4a2f7588 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:05.509298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-33076ecd-b347-4e00-b046-bb2f4a2f7588 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:09:05.509298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:05.509298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:05.509298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:05.509298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:05.509298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:05.509298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:05.509298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:05.509298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:05.509298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:05.509298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:05.509298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:05.509298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:09:05.509298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:09:05.509298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:09:05.510405 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:09:05.510405 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:05.510405 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:05.510405 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:05.510405 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:05.510405 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:05.510405 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:05.510405 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:05.510405 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:05.510405 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:05.510405 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:05.510405 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:05.510405 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:05.510405 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:05.510405 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:05.510405 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:05.510405 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:05.511355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:09:05.511355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:09:05.511355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:09:05.511355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:09:05.515426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 529/1063] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:09:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:09:05.517435 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603e4af-0125-4240-baf4-133ff216f84a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:05.548481 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:05.624396 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603e4af-0125-4240-baf4-133ff216f84a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:05.658086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33021a-a8c1-4f20-a317-d375b1237395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:05.658760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33021a-a8c1-4f20-a317-d375b1237395 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:05.659044 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33021a-a8c1-4f20-a317-d375b1237395 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:05.659346 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33021a-a8c1-4f20-a317-d375b1237395 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:05.660131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33021a-a8c1-4f20-a317-d375b1237395 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:05.660490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33021a-a8c1-4f20-a317-d375b1237395 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:05.662896 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33021a-a8c1-4f20-a317-d375b1237395 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:05.683255 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 535/1064] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:09:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:09:05.858497 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb245d-041a-4955-8e09-737764f18246 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:05.858900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb245d-041a-4955-8e09-737764f18246 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:05.859127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb245d-041a-4955-8e09-737764f18246 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:05.860550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 530/1065] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:09:05.866614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc1490-ae2d-4127-aeab-08dc866dd959 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:05.866907 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc1490-ae2d-4127-aeab-08dc866dd959 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:05.867182 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc1490-ae2d-4127-aeab-08dc866dd959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:05.885761 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-91dc1490-ae2d-4127-aeab-08dc866dd959 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:09:05.886937 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-91dc1490-ae2d-4127-aeab-08dc866dd959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:09:05.955173 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 536/1066] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:09:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:09:05.978713 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb245d-041a-4955-8e09-737764f18246 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:06.082673 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb245d-041a-4955-8e09-737764f18246 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:06.117522 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293df19e-ecb6-4ddd-8437-5008199ec69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:06.118271 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293df19e-ecb6-4ddd-8437-5008199ec69d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:06.118551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293df19e-ecb6-4ddd-8437-5008199ec69d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:06.118820 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293df19e-ecb6-4ddd-8437-5008199ec69d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:06.130863 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293df19e-ecb6-4ddd-8437-5008199ec69d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:09:06.138927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-293df19e-ecb6-4ddd-8437-5008199ec69d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 14:09:06.138927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:06.138927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:06.138927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:06.138927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:06.138927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:06.138927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:06.138927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:06.138927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:06.138927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:06.138927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:06.138927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:06.138927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:09:06.138927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:09:06.138927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:09:06.138927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:09:06.139912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:09:06.139912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:09:06.139912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:09:06.139912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:09:06.139912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:06.139912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:06.139912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:06.139912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:06.139912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:06.139912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:06.139912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:06.139912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:06.139912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:06.139912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:06.139912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:06.139912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:06.140771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:06.140771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:06.140771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:06.140771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:06.140771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:09:06.140771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:09:06.140771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:09:06.140771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:09:06.140771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:09:06.140771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:09:06.140771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 14:09:06.140771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:09:06.140771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 531/1067] 10.222.0.81 () {56 vars in 1120 bytes} [Tue Jun 7 14:09:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:09:06.146823 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc1490-ae2d-4127-aeab-08dc866dd959 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:06.166256 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:06.246655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc1490-ae2d-4127-aeab-08dc866dd959 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:06.280020 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ffa93b-f728-4578-933e-88cd7fca8959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:06.280685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ffa93b-f728-4578-933e-88cd7fca8959 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:06.280967 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ffa93b-f728-4578-933e-88cd7fca8959 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:06.281238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ffa93b-f728-4578-933e-88cd7fca8959 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:06.281980 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ffa93b-f728-4578-933e-88cd7fca8959 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:06.282279 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ffa93b-f728-4578-933e-88cd7fca8959 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:06.283661 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ffa93b-f728-4578-933e-88cd7fca8959 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:06.293871 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 537/1068] 10.222.0.81 () {56 vars in 1115 bytes} [Tue Jun 7 14:09:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:06.299629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293df19e-ecb6-4ddd-8437-5008199ec69d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:06.324878 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:06.397061 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293df19e-ecb6-4ddd-8437-5008199ec69d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:06.429664 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0bb0a-1153-4ffb-a734-bb778abac69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:06.430447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0bb0a-1153-4ffb-a734-bb778abac69e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:06.430726 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0bb0a-1153-4ffb-a734-bb778abac69e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:06.430997 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0bb0a-1153-4ffb-a734-bb778abac69e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:06.442240 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0bb0a-1153-4ffb-a734-bb778abac69e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:06.443709 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0bb0a-1153-4ffb-a734-bb778abac69e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:06.461119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-e5e0bb0a-1153-4ffb-a734-bb778abac69e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:09:06.461119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:06.461119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:06.461119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:06.461119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:06.461119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:06.461119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:06.461119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:06.461119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:06.461119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:06.461119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:06.461119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:06.461119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:09:06.461119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:09:06.461119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:09:06.462190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:09:06.462190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:06.462190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:06.462190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:06.462190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:06.462190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:06.462190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:06.462190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:06.462190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:06.462190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:06.462190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:06.462190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:06.462190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:06.462190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:06.462190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:06.462190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:06.462190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:06.467781 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:09:06.467781 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:09:06.467781 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:09:06.467781 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:09:06.467781 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 532/1069] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:09:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:09:06.468083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ffa93b-f728-4578-933e-88cd7fca8959 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:06.546708 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:06.569875 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ffa93b-f728-4578-933e-88cd7fca8959 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:06.603733 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5247a8b-59b3-475e-b157-487dd33069bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:06.604388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5247a8b-59b3-475e-b157-487dd33069bc None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:06.604665 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5247a8b-59b3-475e-b157-487dd33069bc None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:06.604937 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5247a8b-59b3-475e-b157-487dd33069bc None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:06.605655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5247a8b-59b3-475e-b157-487dd33069bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:06.605954 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5247a8b-59b3-475e-b157-487dd33069bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:06.607332 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5247a8b-59b3-475e-b157-487dd33069bc None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:06.623974 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 538/1070] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:06.625088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0bb0a-1153-4ffb-a734-bb778abac69e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:06.701245 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:06.724411 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0bb0a-1153-4ffb-a734-bb778abac69e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:06.758090 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a89091-2674-411f-b11c-4bc9aee21d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:06.759178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a89091-2674-411f-b11c-4bc9aee21d3d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:06.759508 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a89091-2674-411f-b11c-4bc9aee21d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:06.759780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a89091-2674-411f-b11c-4bc9aee21d3d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:06.767368 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a89091-2674-411f-b11c-4bc9aee21d3d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:06.768802 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a89091-2674-411f-b11c-4bc9aee21d3d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:06.783040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-62a89091-2674-411f-b11c-4bc9aee21d3d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:09:06.783040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:06.783040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:06.783040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:06.783040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:06.783040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:06.783040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:06.783040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:06.783040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:06.783040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:06.783040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:06.783040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:06.783040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:09:06.783040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:09:06.783040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:09:06.783040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:09:06.784142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:06.784142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:06.784142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:09:06.784142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:09:06.784142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:09:06.784142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:09:06.784142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:06.784142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:06.784142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:06.784142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:06.784142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:06.784142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:06.784142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:06.784142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:06.784142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:06.784142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:06.784142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:06.784142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:06.785127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:06.785127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:06.785127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:09:06.785127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:09:06.785127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:09:06.785127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:09:06.785127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:09:06.785127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:09:06.785127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:09:06.785127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:09:06.785127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 533/1071] 10.222.0.81 () {56 vars in 1093 bytes} [Tue Jun 7 14:09:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:09:06.790163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5247a8b-59b3-475e-b157-487dd33069bc None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:06.889197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5247a8b-59b3-475e-b157-487dd33069bc None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:06.923013 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26204645-948e-4c20-91e2-b98aa57fde9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:06.923715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26204645-948e-4c20-91e2-b98aa57fde9a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:06.924004 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26204645-948e-4c20-91e2-b98aa57fde9a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:06.924269 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26204645-948e-4c20-91e2-b98aa57fde9a None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:06.925102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26204645-948e-4c20-91e2-b98aa57fde9a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:06.925412 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26204645-948e-4c20-91e2-b98aa57fde9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:06.927653 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26204645-948e-4c20-91e2-b98aa57fde9a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:06.936657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:06.954202 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 539/1072] 10.222.0.81 () {56 vars in 1133 bytes} [Tue Jun 7 14:09:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:06.955335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a89091-2674-411f-b11c-4bc9aee21d3d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:07.061205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a89091-2674-411f-b11c-4bc9aee21d3d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:07.093756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb29c99-999f-462a-9df5-9a71b38ff470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:07.094538 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb29c99-999f-462a-9df5-9a71b38ff470 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:07.094817 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb29c99-999f-462a-9df5-9a71b38ff470 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:07.095090 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb29c99-999f-462a-9df5-9a71b38ff470 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:07.097714 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:07.106125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb29c99-999f-462a-9df5-9a71b38ff470 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:07.107584 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb29c99-999f-462a-9df5-9a71b38ff470 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:07.124782 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-9bb29c99-999f-462a-9df5-9a71b38ff470 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:09:07.124782 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:07.124782 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:07.124782 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:07.124782 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:07.124782 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:07.124782 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:07.124782 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:07.124782 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:07.124782 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:07.124782 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:07.124782 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:07.124782 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:09:07.124782 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:09:07.124782 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:09:07.125747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:09:07.125747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:07.125747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:07.125747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:07.125747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:07.125747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:07.125747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:07.125747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:07.125747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:07.125747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:07.125747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:07.125747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:07.125747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:07.125747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:07.125747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:07.125747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:07.125747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:07.126646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:09:07.126646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:09:07.126646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:09:07.126646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:09:07.126646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 534/1073] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:09:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:09:07.132522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26204645-948e-4c20-91e2-b98aa57fde9a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:07.229377 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26204645-948e-4c20-91e2-b98aa57fde9a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:07.263051 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0987e2ed-d25a-4a59-bf45-13c1c71b18ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:07.263737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987e2ed-d25a-4a59-bf45-13c1c71b18ee None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:07.264148 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987e2ed-d25a-4a59-bf45-13c1c71b18ee None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:07.264424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987e2ed-d25a-4a59-bf45-13c1c71b18ee None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:07.265144 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0987e2ed-d25a-4a59-bf45-13c1c71b18ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:07.265446 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0987e2ed-d25a-4a59-bf45-13c1c71b18ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:07.266808 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0987e2ed-d25a-4a59-bf45-13c1c71b18ee None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:07.278084 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 540/1074] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:07.278234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b5413-2d59-47a6-b1ac-944548ed6492 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:07.278326 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b5413-2d59-47a6-b1ac-944548ed6492 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:07.278595 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b5413-2d59-47a6-b1ac-944548ed6492 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:07.280078 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 535/1075] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:09:07.283522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0987e2ed-d25a-4a59-bf45-13c1c71b18ee None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:07.285087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574bc9b-f1c3-4d7d-a885-667f3e2db8a1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:07.285988 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574bc9b-f1c3-4d7d-a885-667f3e2db8a1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:07.286534 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574bc9b-f1c3-4d7d-a885-667f3e2db8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:07.303536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9574bc9b-f1c3-4d7d-a885-667f3e2db8a1 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:09:07.304727 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9574bc9b-f1c3-4d7d-a885-667f3e2db8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:09:07.341911 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:07.371133 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 536/1076] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:09:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:09:07.385753 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0987e2ed-d25a-4a59-bf45-13c1c71b18ee None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:07.394023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9574bc9b-f1c3-4d7d-a885-667f3e2db8a1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:07.420128 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de750d-1f72-4551-b858-a71ac35dd4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:07.420920 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de750d-1f72-4551-b858-a71ac35dd4a3 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:07.421204 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de750d-1f72-4551-b858-a71ac35dd4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:07.421474 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de750d-1f72-4551-b858-a71ac35dd4a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:07.428864 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de750d-1f72-4551-b858-a71ac35dd4a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:07.430299 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de750d-1f72-4551-b858-a71ac35dd4a3 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:07.444863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-c3de750d-1f72-4551-b858-a71ac35dd4a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:09:07.444863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:07.444863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:07.444863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:07.444863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:07.444863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:07.444863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:07.444863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:07.444863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:07.444863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:07.444863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:07.444863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:07.444863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:09:07.444863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:09:07.444863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:09:07.445944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:09:07.445944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:07.445944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:07.445944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:07.445944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:07.445944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:07.445944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:07.445944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:07.445944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:07.445944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:07.445944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:07.445944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:07.445944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:07.445944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:07.445944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:07.445944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:07.445944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:07.446875 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:09:07.446875 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:09:07.446875 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:09:07.446875 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:09:07.446875 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:09:07.446875 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:09:07.446875 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:09:07.446875 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:09:07.446875 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 541/1077] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:09:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:09:07.451727 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de750d-1f72-4551-b858-a71ac35dd4a3 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:07.497978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9574bc9b-f1c3-4d7d-a885-667f3e2db8a1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:07.514953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:07.531925 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54d2f8-cf85-440a-952c-a044dee8a016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:07.532729 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54d2f8-cf85-440a-952c-a044dee8a016 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:07.533016 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54d2f8-cf85-440a-952c-a044dee8a016 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:07.533286 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54d2f8-cf85-440a-952c-a044dee8a016 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:07.543931 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54d2f8-cf85-440a-952c-a044dee8a016 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:09:07.552322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-4d54d2f8-cf85-440a-952c-a044dee8a016 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:09:07.552322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:07.552322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:07.552322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:07.552322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:07.552322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:07.552322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:07.552322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:07.552322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:07.552322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:07.552322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:07.552322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:07.552322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:09:07.552322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:09:07.552322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:09:07.552322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:09:07.553357 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:09:07.553357 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:09:07.553357 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:09:07.553357 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:09:07.553357 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:07.553357 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:07.553357 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:07.553357 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:07.553357 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:07.553357 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:07.553357 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:07.553357 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:07.553357 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:07.553357 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:07.553357 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:07.553357 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:07.554223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:07.554223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:07.554223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:07.554223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:07.554223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:09:07.554223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:09:07.554223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:09:07.554223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:09:07.554223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:09:07.554223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:09:07.554223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:09:07.554223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:09:07.558471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 537/1078] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:09:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:09:07.559318 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54d2f8-cf85-440a-952c-a044dee8a016 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:07.559506 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de750d-1f72-4551-b858-a71ac35dd4a3 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:07.575935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:07.594118 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb685f-2ce9-4f28-9674-09b30d7dba9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:07.594772 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb685f-2ce9-4f28-9674-09b30d7dba9b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:07.595049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb685f-2ce9-4f28-9674-09b30d7dba9b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:07.595348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb685f-2ce9-4f28-9674-09b30d7dba9b None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:07.596123 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eb685f-2ce9-4f28-9674-09b30d7dba9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:07.596427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eb685f-2ce9-4f28-9674-09b30d7dba9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:07.598350 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eb685f-2ce9-4f28-9674-09b30d7dba9b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:07.610017 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 542/1079] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:09:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:09:07.616896 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb685f-2ce9-4f28-9674-09b30d7dba9b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:07.659708 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54d2f8-cf85-440a-952c-a044dee8a016 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:07.693299 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d360b39-6f3e-4938-be05-34b95475a0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:07.693966 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d360b39-6f3e-4938-be05-34b95475a0d4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:07.694247 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d360b39-6f3e-4938-be05-34b95475a0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:07.694519 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d360b39-6f3e-4938-be05-34b95475a0d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:07.695283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d360b39-6f3e-4938-be05-34b95475a0d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:07.695590 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d360b39-6f3e-4938-be05-34b95475a0d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:07.696967 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d360b39-6f3e-4938-be05-34b95475a0d4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:07.704883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 538/1080] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:09:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:07.711595 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d360b39-6f3e-4938-be05-34b95475a0d4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:07.715616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb685f-2ce9-4f28-9674-09b30d7dba9b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:07.751684 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:07.753623 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d314aa-d488-4a73-af21-a1bdbcfcf7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:07.754685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d314aa-d488-4a73-af21-a1bdbcfcf7a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:07.754966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d314aa-d488-4a73-af21-a1bdbcfcf7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:07.755263 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d314aa-d488-4a73-af21-a1bdbcfcf7a4 None admin] PATH_INFO: `/v3/projects/f67bad575f8b4a47bee7c314497cb2fe/users/2ae726fbc9cf40e68c7587552a118a7a/roles/855dcf6914224d5c907aa41ec8c2e0c4` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:07.757611 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d314aa-d488-4a73-af21-a1bdbcfcf7a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f67bad575f8b4a47bee7c314497cb2fe, user_id=2ae726fbc9cf40e68c7587552a118a7a, role_id=855dcf6914224d5c907aa41ec8c2e0c4)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:07.759130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d314aa-d488-4a73-af21-a1bdbcfcf7a4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:07.772813 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 543/1081] 10.222.0.81 () {58 vars in 1439 bytes} [Tue Jun 7 14:09:07 2022] PUT /identity/v3/projects/f67bad575f8b4a47bee7c314497cb2fe/users/2ae726fbc9cf40e68c7587552a118a7a/roles/855dcf6914224d5c907aa41ec8c2e0c4 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:09:07.813617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d360b39-6f3e-4938-be05-34b95475a0d4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:07.850747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e360fc0-4c90-4652-8a3b-19d66272dca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:07.851669 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e360fc0-4c90-4652-8a3b-19d66272dca7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:07.851953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e360fc0-4c90-4652-8a3b-19d66272dca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:07.852225 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e360fc0-4c90-4652-8a3b-19d66272dca7 None admin] PATH_INFO: `/v3/users/0bf1d2d1c3344504a1eef850b1a420b4` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:07.853473 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e360fc0-4c90-4652-8a3b-19d66272dca7 None admin] RBAC: Authorizing `identity:get_user(user_id=0bf1d2d1c3344504a1eef850b1a420b4)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:07.854963 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e360fc0-4c90-4652-8a3b-19d66272dca7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:07.856724 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 539/1082] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:09:07 2022] GET /identity/v3/users/0bf1d2d1c3344504a1eef850b1a420b4 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:07.862732 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d314aa-d488-4a73-af21-a1bdbcfcf7a4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:07.901547 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:07.972293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d314aa-d488-4a73-af21-a1bdbcfcf7a4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:08.007536 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9b29c-2256-4b60-911f-d7dc194650a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:08.008315 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9b29c-2256-4b60-911f-d7dc194650a7 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:08.008594 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9b29c-2256-4b60-911f-d7dc194650a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:08.008865 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9b29c-2256-4b60-911f-d7dc194650a7 None admin] PATH_INFO: `/v3/projects/288242db145547d480bf1522f1d0440f` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:08.010025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be9b29c-2256-4b60-911f-d7dc194650a7 None admin] RBAC: Authorizing `identity:get_project(project_id=288242db145547d480bf1522f1d0440f)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:08.011467 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be9b29c-2256-4b60-911f-d7dc194650a7 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:08.013195 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 544/1083] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:09:07 2022] GET /identity/v3/projects/288242db145547d480bf1522f1d0440f => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:08.019152 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e360fc0-4c90-4652-8a3b-19d66272dca7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:08.131278 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e360fc0-4c90-4652-8a3b-19d66272dca7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:08.140124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:08.166770 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd124d-623d-4671-b75f-8838a2fc1dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:08.167698 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd124d-623d-4671-b75f-8838a2fc1dac None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:08.167979 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd124d-623d-4671-b75f-8838a2fc1dac None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:08.168248 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd124d-623d-4671-b75f-8838a2fc1dac None admin] PATH_INFO: `/v3/projects/288242db145547d480bf1522f1d0440f/users/0bf1d2d1c3344504a1eef850b1a420b4/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:08.170171 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd124d-623d-4671-b75f-8838a2fc1dac None admin] RBAC: Authorizing `identity:create_grant(project_id=288242db145547d480bf1522f1d0440f, user_id=0bf1d2d1c3344504a1eef850b1a420b4, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:08.171620 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd124d-623d-4671-b75f-8838a2fc1dac None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:08.184306 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 540/1084] 10.222.0.81 () {58 vars in 1439 bytes} [Tue Jun 7 14:09:08 2022] PUT /identity/v3/projects/288242db145547d480bf1522f1d0440f/users/0bf1d2d1c3344504a1eef850b1a420b4/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:09:10.025507 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30942880-cf93-4a7b-b19f-c1b51bbb2445 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:10.026044 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30942880-cf93-4a7b-b19f-c1b51bbb2445 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:10.026044 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30942880-cf93-4a7b-b19f-c1b51bbb2445 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:10.027468 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 545/1085] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:09:10.032822 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10570e73-4590-4c5f-a8db-ba727414bf1d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:10.033227 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10570e73-4590-4c5f-a8db-ba727414bf1d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:10.033504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10570e73-4590-4c5f-a8db-ba727414bf1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:10.050198 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-10570e73-4590-4c5f-a8db-ba727414bf1d None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:09:10.052228 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-10570e73-4590-4c5f-a8db-ba727414bf1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:09:10.121788 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 541/1086] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:09:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:09:10.146507 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30942880-cf93-4a7b-b19f-c1b51bbb2445 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:10.248919 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30942880-cf93-4a7b-b19f-c1b51bbb2445 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:10.251201 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:10.282350 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70c0fb-8ebc-45b7-9948-a2bce13e6352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:10.283080 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70c0fb-8ebc-45b7-9948-a2bce13e6352 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:10.283398 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70c0fb-8ebc-45b7-9948-a2bce13e6352 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:10.283669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70c0fb-8ebc-45b7-9948-a2bce13e6352 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:10.294146 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f70c0fb-8ebc-45b7-9948-a2bce13e6352 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:09:10.303022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-4f70c0fb-8ebc-45b7-9948-a2bce13e6352 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 14:09:10.303022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:10.303022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:10.303022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:10.303022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:10.303022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:10.303022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:10.303022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:10.303022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:10.303022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:10.303022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:10.303022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:10.303022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:09:10.303022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:09:10.303022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:09:10.303022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:09:10.304008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:09:10.304008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:09:10.304008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:09:10.304008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:09:10.304008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:10.304008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:10.304008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:10.304008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:10.304008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:10.304008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:10.304008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:10.304008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:10.304008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:10.304008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:10.304008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:10.304008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:10.304881 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:10.304881 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:10.304881 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:10.304881 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:10.304881 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:09:10.304881 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:09:10.304881 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:09:10.304881 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:09:10.304881 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:09:10.304881 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:09:10.304881 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 14:09:10.304881 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:09:10.304881 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 546/1087] 10.222.0.81 () {56 vars in 1120 bytes} [Tue Jun 7 14:09:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:09:10.310187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10570e73-4590-4c5f-a8db-ba727414bf1d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:10.404358 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:10.408278 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10570e73-4590-4c5f-a8db-ba727414bf1d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:10.440887 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa164799-fe0c-409e-b830-652356b23ba6 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:10.441170 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa164799-fe0c-409e-b830-652356b23ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:10.441440 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa164799-fe0c-409e-b830-652356b23ba6 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:10.443426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 547/1088] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:09:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:09:10.444337 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b720ad-72b1-40e4-ac4f-4fd37e9f9346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:10.445013 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b720ad-72b1-40e4-ac4f-4fd37e9f9346 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:10.445294 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b720ad-72b1-40e4-ac4f-4fd37e9f9346 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:10.445568 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b720ad-72b1-40e4-ac4f-4fd37e9f9346 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:10.446322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b720ad-72b1-40e4-ac4f-4fd37e9f9346 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:10.446626 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b720ad-72b1-40e4-ac4f-4fd37e9f9346 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:10.448235 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cec859-21c5-406a-8927-c6a305e2d67b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:10.448353 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b720ad-72b1-40e4-ac4f-4fd37e9f9346 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:10.448517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cec859-21c5-406a-8927-c6a305e2d67b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:10.448791 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cec859-21c5-406a-8927-c6a305e2d67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:10.460888 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 542/1089] 10.222.0.81 () {56 vars in 1115 bytes} [Tue Jun 7 14:09:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:10.461928 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b720ad-72b1-40e4-ac4f-4fd37e9f9346 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:10.466879 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-52cec859-21c5-406a-8927-c6a305e2d67b None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:09:10.468072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-52cec859-21c5-406a-8927-c6a305e2d67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:09:10.537553 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 548/1090] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:09:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:09:10.560576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b720ad-72b1-40e4-ac4f-4fd37e9f9346 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:10.561595 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cec859-21c5-406a-8927-c6a305e2d67b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:10.593719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a48be-1f7b-4d90-bbd1-869106d4deb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:10.594511 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a48be-1f7b-4d90-bbd1-869106d4deb9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:10.594791 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a48be-1f7b-4d90-bbd1-869106d4deb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:10.595065 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a48be-1f7b-4d90-bbd1-869106d4deb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:10.606252 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a48be-1f7b-4d90-bbd1-869106d4deb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:10.607720 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a48be-1f7b-4d90-bbd1-869106d4deb9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:10.625660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-044a48be-1f7b-4d90-bbd1-869106d4deb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:09:10.625660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:10.625660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:10.625660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:10.625660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:10.625660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:10.625660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:10.625660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:10.625660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:10.625660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:10.625660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:10.625660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:10.625660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:09:10.625660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:09:10.625660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:09:10.626789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:09:10.626789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:10.626789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:10.626789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:10.626789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:10.626789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:10.626789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:10.626789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:10.626789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:10.626789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:10.626789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:10.626789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:10.626789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:10.626789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:10.626789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:10.626789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:10.626789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:10.627755 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:09:10.627755 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:09:10.627755 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:09:10.627755 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:09:10.627755 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 543/1091] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:09:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:09:10.632782 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a48be-1f7b-4d90-bbd1-869106d4deb9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:10.648781 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:10.655827 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:10.664131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cec859-21c5-406a-8927-c6a305e2d67b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:10.697120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471e3db-0faa-42f1-b750-abb9c6435d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:10.697903 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471e3db-0faa-42f1-b750-abb9c6435d2d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:10.698182 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471e3db-0faa-42f1-b750-abb9c6435d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:10.698452 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471e3db-0faa-42f1-b750-abb9c6435d2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:10.709302 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471e3db-0faa-42f1-b750-abb9c6435d2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:09:10.716928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-4471e3db-0faa-42f1-b750-abb9c6435d2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:09:10.716928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:10.716928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:10.716928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:10.716928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:10.716928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:10.716928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:10.716928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:10.716928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:10.716928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:10.716928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:10.716928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:10.716928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:09:10.716928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:09:10.716928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:09:10.716928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:09:10.718057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:09:10.718057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:09:10.718057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:09:10.718057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:09:10.718057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:10.718057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:10.718057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:10.718057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:10.718057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:10.718057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:10.718057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:10.718057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:10.718057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:10.718057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:10.718057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:10.718057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:10.719117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:10.719117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:10.719117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:10.719117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:10.719117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:09:10.719117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:09:10.719117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:09:10.719117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:09:10.719117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:09:10.719117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:09:10.719117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:09:10.719117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:09:10.719117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 549/1092] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:09:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:09:10.724663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471e3db-0faa-42f1-b750-abb9c6435d2d None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:10.736198 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a48be-1f7b-4d90-bbd1-869106d4deb9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:10.769206 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab40864-4ea6-4cef-9c8b-a0308a4f790c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:10.770554 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab40864-4ea6-4cef-9c8b-a0308a4f790c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:10.770839 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab40864-4ea6-4cef-9c8b-a0308a4f790c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:10.771109 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab40864-4ea6-4cef-9c8b-a0308a4f790c None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:10.771853 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab40864-4ea6-4cef-9c8b-a0308a4f790c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:10.772156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab40864-4ea6-4cef-9c8b-a0308a4f790c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:10.773512 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab40864-4ea6-4cef-9c8b-a0308a4f790c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:10.786113 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 544/1093] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:09:10.791700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab40864-4ea6-4cef-9c8b-a0308a4f790c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:10.824157 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471e3db-0faa-42f1-b750-abb9c6435d2d None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:10.857394 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f4f3d-09d8-440d-baef-43fc2fda0a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:10.858052 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f4f3d-09d8-440d-baef-43fc2fda0a7d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:10.858329 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f4f3d-09d8-440d-baef-43fc2fda0a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:10.858599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f4f3d-09d8-440d-baef-43fc2fda0a7d None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:10.859368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f4f3d-09d8-440d-baef-43fc2fda0a7d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:10.859673 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f4f3d-09d8-440d-baef-43fc2fda0a7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:10.859910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:10.861064 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f4f3d-09d8-440d-baef-43fc2fda0a7d None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:10.869647 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 550/1094] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:09:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:09:10.875372 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f4f3d-09d8-440d-baef-43fc2fda0a7d None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:10.889619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab40864-4ea6-4cef-9c8b-a0308a4f790c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:10.892454 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:10.925306 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02125e2-c657-4ad7-81cc-cecf60ccf1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:10.926232 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02125e2-c657-4ad7-81cc-cecf60ccf1ed None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:10.926653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02125e2-c657-4ad7-81cc-cecf60ccf1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:10.926937 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02125e2-c657-4ad7-81cc-cecf60ccf1ed None admin] PATH_INFO: `/v3/users/nova` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:10.933928 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02125e2-c657-4ad7-81cc-cecf60ccf1ed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:10.935416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02125e2-c657-4ad7-81cc-cecf60ccf1ed None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:10.949415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-b02125e2-c657-4ad7-81cc-cecf60ccf1ed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:09:10.949415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:10.949415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:10.949415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:10.949415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:10.949415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:10.949415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:10.949415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:10.949415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:10.949415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:10.949415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:10.949415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:10.949415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:09:10.949415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:09:10.949415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:09:10.949415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:09:10.950565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:10.950565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:10.950565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:09:10.950565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:09:10.950565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:09:10.950565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:09:10.950565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:10.950565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:10.950565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:10.950565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:10.950565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:10.950565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:10.950565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:10.950565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:10.950565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:10.950565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:10.950565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:10.950565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:10.951603 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:10.951603 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:10.951603 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:09:10.951603 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:09:10.951603 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:09:10.951603 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:09:10.951603 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:09:10.951603 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:09:10.951603 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:09:10.951603 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:09:10.951603 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 545/1095] 10.222.0.81 () {56 vars in 1093 bytes} [Tue Jun 7 14:09:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:09:10.956223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02125e2-c657-4ad7-81cc-cecf60ccf1ed None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:10.978887 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f4f3d-09d8-440d-baef-43fc2fda0a7d None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:11.014377 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac1e93-6bed-47cc-a681-f2659237b765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:11.015273 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac1e93-6bed-47cc-a681-f2659237b765 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:11.015553 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac1e93-6bed-47cc-a681-f2659237b765 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:11.015828 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac1e93-6bed-47cc-a681-f2659237b765 None admin] PATH_INFO: `/v3/users/0bf1d2d1c3344504a1eef850b1a420b4` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:11.017072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac1e93-6bed-47cc-a681-f2659237b765 None admin] RBAC: Authorizing `identity:get_user(user_id=0bf1d2d1c3344504a1eef850b1a420b4)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:11.018485 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac1e93-6bed-47cc-a681-f2659237b765 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:11.021054 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 551/1096] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:09:10 2022] GET /identity/v3/users/0bf1d2d1c3344504a1eef850b1a420b4 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:11.026649 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac1e93-6bed-47cc-a681-f2659237b765 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:11.067101 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02125e2-c657-4ad7-81cc-cecf60ccf1ed None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:11.099166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:11.105679 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883591a5-732a-46ee-8aa6-abd90ea8bd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:11.106359 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883591a5-732a-46ee-8aa6-abd90ea8bd0e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:11.106639 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883591a5-732a-46ee-8aa6-abd90ea8bd0e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:11.106914 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883591a5-732a-46ee-8aa6-abd90ea8bd0e None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:11.107787 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883591a5-732a-46ee-8aa6-abd90ea8bd0e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:11.108098 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883591a5-732a-46ee-8aa6-abd90ea8bd0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:11.109525 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883591a5-732a-46ee-8aa6-abd90ea8bd0e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:11.119985 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:11.127080 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 546/1097] 10.222.0.81 () {56 vars in 1133 bytes} [Tue Jun 7 14:09:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:09:11.133320 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883591a5-732a-46ee-8aa6-abd90ea8bd0e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:11.137240 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac1e93-6bed-47cc-a681-f2659237b765 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:11.176226 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9094e6e-5a1f-47bd-88c2-e4dda3dbe82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:11.176886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9094e6e-5a1f-47bd-88c2-e4dda3dbe82a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:11.177167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9094e6e-5a1f-47bd-88c2-e4dda3dbe82a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:11.177444 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9094e6e-5a1f-47bd-88c2-e4dda3dbe82a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:11.231084 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883591a5-732a-46ee-8aa6-abd90ea8bd0e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:11.244780 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9094e6e-5a1f-47bd-88c2-e4dda3dbe82a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:11.246225 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9094e6e-5a1f-47bd-88c2-e4dda3dbe82a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:11.265357 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2df4d-e475-42ce-b531-7482ca1fd12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:11.265949 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2df4d-e475-42ce-b531-7482ca1fd12a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:11.266229 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2df4d-e475-42ce-b531-7482ca1fd12a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:11.266502 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2df4d-e475-42ce-b531-7482ca1fd12a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:11.323642 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:11.327632 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:11.335112 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2df4d-e475-42ce-b531-7482ca1fd12a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:11.336615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2df4d-e475-42ce-b531-7482ca1fd12a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:11.343880 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 552/1098] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:09:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 07 14:09:11.350378 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9094e6e-5a1f-47bd-88c2-e4dda3dbe82a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:11.439126 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 547/1099] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:09:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 07 14:09:11.445671 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2df4d-e475-42ce-b531-7482ca1fd12a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:11.448866 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9094e6e-5a1f-47bd-88c2-e4dda3dbe82a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:11.482287 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6e743-0954-4429-beca-ec84c31590a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:11.483142 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6e743-0954-4429-beca-ec84c31590a4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:11.483454 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6e743-0954-4429-beca-ec84c31590a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:11.483726 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6e743-0954-4429-beca-ec84c31590a4 None admin] PATH_INFO: `/v3/projects/288242db145547d480bf1522f1d0440f` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:11.484894 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6e743-0954-4429-beca-ec84c31590a4 None admin] RBAC: Authorizing `identity:get_project(project_id=288242db145547d480bf1522f1d0440f)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:11.486310 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6e743-0954-4429-beca-ec84c31590a4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:11.488042 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 553/1100] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:09:11 2022] GET /identity/v3/projects/288242db145547d480bf1522f1d0440f => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:09:11.495160 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6e743-0954-4429-beca-ec84c31590a4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:11.533403 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:11.543242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2df4d-e475-42ce-b531-7482ca1fd12a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:11.547140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:11.577278 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edf24a-4239-433f-a78a-6a931495fba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:11.577992 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edf24a-4239-433f-a78a-6a931495fba9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:11.578272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edf24a-4239-433f-a78a-6a931495fba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:11.578541 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edf24a-4239-433f-a78a-6a931495fba9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:11.588445 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7edf24a-4239-433f-a78a-6a931495fba9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:11.589892 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7edf24a-4239-433f-a78a-6a931495fba9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:11.594682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6e743-0954-4429-beca-ec84c31590a4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:11.606797 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-a7edf24a-4239-433f-a78a-6a931495fba9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:09:11.606797 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:11.606797 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:11.606797 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:11.606797 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:11.606797 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:11.606797 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:11.606797 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:11.606797 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:11.606797 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:11.606797 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:11.606797 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:11.606797 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:09:11.606797 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:09:11.606797 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:09:11.607827 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:09:11.607827 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:11.607827 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:11.607827 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:11.607827 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:11.607827 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:11.607827 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:11.607827 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:11.607827 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:11.607827 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:11.607827 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:11.607827 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:11.607827 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:11.607827 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:11.607827 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:11.607827 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:11.607827 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:11.608922 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:09:11.608922 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:09:11.608922 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:09:11.608922 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:09:11.608922 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 548/1101] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:09:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:09:11.614101 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edf24a-4239-433f-a78a-6a931495fba9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:11.628506 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ce5af-572c-432f-99f8-a525b8315363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:11.629753 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ce5af-572c-432f-99f8-a525b8315363 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:11.630026 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ce5af-572c-432f-99f8-a525b8315363 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:11.630300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ce5af-572c-432f-99f8-a525b8315363 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:11.631122 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ce5af-572c-432f-99f8-a525b8315363 None admin] RBAC: Adding query filter params (role.id=82bbe3651b9b4f378fd150d93357b2b1, scope.project.id=288242db145547d480bf1522f1d0440f, user.id=0bf1d2d1c3344504a1eef850b1a420b4) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:11.631464 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ce5af-572c-432f-99f8-a525b8315363 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:11.632878 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ce5af-572c-432f-99f8-a525b8315363 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:11.638611 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 554/1102] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:09:11 2022] GET /identity/v3/role_assignments?user.id=0bf1d2d1c3344504a1eef850b1a420b4&scope.project.id=288242db145547d480bf1522f1d0440f&role.id=82bbe3651b9b4f378fd150d93357b2b1 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:09:11.717783 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edf24a-4239-433f-a78a-6a931495fba9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:11.755533 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa886fa7-a87a-4671-b483-88762c0ee024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:11.756206 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa886fa7-a87a-4671-b483-88762c0ee024 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:11.756483 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa886fa7-a87a-4671-b483-88762c0ee024 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:11.756754 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa886fa7-a87a-4671-b483-88762c0ee024 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:11.757494 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa886fa7-a87a-4671-b483-88762c0ee024 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:11.757794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa886fa7-a87a-4671-b483-88762c0ee024 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:11.759177 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa886fa7-a87a-4671-b483-88762c0ee024 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:11.771694 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 549/1103] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:09:11.777470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ce5af-572c-432f-99f8-a525b8315363 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:11.880964 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ce5af-572c-432f-99f8-a525b8315363 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:11.897378 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:11.915630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c02fa-1560-4a75-bef1-b9495639cf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:11.916426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c02fa-1560-4a75-bef1-b9495639cf16 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:11.916710 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c02fa-1560-4a75-bef1-b9495639cf16 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:11.916984 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c02fa-1560-4a75-bef1-b9495639cf16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:11.923723 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c02fa-1560-4a75-bef1-b9495639cf16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:11.925152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c02fa-1560-4a75-bef1-b9495639cf16 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:11.939855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-798c02fa-1560-4a75-bef1-b9495639cf16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:09:11.939855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:11.939855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:11.939855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:11.939855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:11.939855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:11.939855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:11.939855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:11.939855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:11.939855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:11.939855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:11.939855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:11.939855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:09:11.939855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:09:11.939855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:09:11.940781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:09:11.940781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:11.940781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:11.940781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:11.940781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:11.940781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:11.940781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:11.940781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:11.940781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:11.940781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:11.940781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:11.940781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:11.940781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:11.940781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:11.940781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:11.940781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:11.940781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:11.941844 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:09:11.941844 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:09:11.941844 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:09:11.941844 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:09:11.941844 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:09:11.941844 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:09:11.941844 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:09:11.941844 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:09:11.941844 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 555/1104] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:09:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:09:11.946827 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa886fa7-a87a-4671-b483-88762c0ee024 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:11.949697 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:12.045880 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa886fa7-a87a-4671-b483-88762c0ee024 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:12.079121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a36e48-2aa8-4b7b-826f-40c92f443220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:12.079819 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a36e48-2aa8-4b7b-826f-40c92f443220 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:12.080078 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a36e48-2aa8-4b7b-826f-40c92f443220 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:12.080351 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a36e48-2aa8-4b7b-826f-40c92f443220 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:12.081137 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a36e48-2aa8-4b7b-826f-40c92f443220 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:12.081442 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a36e48-2aa8-4b7b-826f-40c92f443220 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:12.082888 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a36e48-2aa8-4b7b-826f-40c92f443220 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:12.094782 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 550/1105] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:09:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:12.101439 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c02fa-1560-4a75-bef1-b9495639cf16 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:12.207812 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c02fa-1560-4a75-bef1-b9495639cf16 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:12.241485 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce63ca-ba99-4b16-857b-44db7937b37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:12.242138 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce63ca-ba99-4b16-857b-44db7937b37b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:12.242424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce63ca-ba99-4b16-857b-44db7937b37b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:12.242702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce63ca-ba99-4b16-857b-44db7937b37b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:12.243574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce63ca-ba99-4b16-857b-44db7937b37b None admin] RBAC: Adding query filter params (role.id=855dcf6914224d5c907aa41ec8c2e0c4, scope.project.id=f67bad575f8b4a47bee7c314497cb2fe, user.id=2ae726fbc9cf40e68c7587552a118a7a) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:12.243891 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce63ca-ba99-4b16-857b-44db7937b37b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:12.245578 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce63ca-ba99-4b16-857b-44db7937b37b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:12.251321 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 556/1106] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:09:12 2022] GET /identity/v3/role_assignments?user.id=2ae726fbc9cf40e68c7587552a118a7a&scope.project.id=f67bad575f8b4a47bee7c314497cb2fe&role.id=855dcf6914224d5c907aa41ec8c2e0c4 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:13.904030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d28901-a328-4914-9a66-8591101bd958 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:13.904585 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d28901-a328-4914-9a66-8591101bd958 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:13.904585 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d28901-a328-4914-9a66-8591101bd958 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:13.905935 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 551/1107] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:09:13.911019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363144ea-d809-4f04-b495-2a05c72cad6f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:13.911435 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363144ea-d809-4f04-b495-2a05c72cad6f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:13.911715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363144ea-d809-4f04-b495-2a05c72cad6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:13.928708 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-363144ea-d809-4f04-b495-2a05c72cad6f None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:09:13.929878 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-363144ea-d809-4f04-b495-2a05c72cad6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:09:13.963177 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:13.995857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 557/1108] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:09:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:09:14.020523 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d28901-a328-4914-9a66-8591101bd958 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:14.080834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:14.119553 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d28901-a328-4914-9a66-8591101bd958 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:14.152848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e67d54-0649-4153-91b7-2577ce288d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:14.153640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e67d54-0649-4153-91b7-2577ce288d60 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:14.153925 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e67d54-0649-4153-91b7-2577ce288d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:14.154255 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e67d54-0649-4153-91b7-2577ce288d60 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:14.155439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e67d54-0649-4153-91b7-2577ce288d60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:14.156881 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e67d54-0649-4153-91b7-2577ce288d60 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:14.158513 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 552/1109] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:09:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:14.164268 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363144ea-d809-4f04-b495-2a05c72cad6f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:14.263303 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363144ea-d809-4f04-b495-2a05c72cad6f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:14.295965 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d592c4a-e320-430a-9eea-f5d515c0b666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:14.296628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d592c4a-e320-430a-9eea-f5d515c0b666 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:14.296911 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d592c4a-e320-430a-9eea-f5d515c0b666 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:14.297186 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d592c4a-e320-430a-9eea-f5d515c0b666 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:14.297883 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d592c4a-e320-430a-9eea-f5d515c0b666 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:14.299342 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d592c4a-e320-430a-9eea-f5d515c0b666 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:14.320071 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 558/1110] 10.222.0.81 () {60 vars in 1132 bytes} [Tue Jun 7 14:09:14 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:09:16.583397 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc0e46-f105-4020-91c8-a311732ba405 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:16.583956 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc0e46-f105-4020-91c8-a311732ba405 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:16.583956 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc0e46-f105-4020-91c8-a311732ba405 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:16.585291 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 553/1111] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:09:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:09:16.590559 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c6ca1-6a19-4285-b090-538721843f76 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:16.590979 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c6ca1-6a19-4285-b090-538721843f76 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:16.591100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c6ca1-6a19-4285-b090-538721843f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:16.608311 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-e89c6ca1-6a19-4285-b090-538721843f76 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:09:16.609480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e89c6ca1-6a19-4285-b090-538721843f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:09:16.630150 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:16.676464 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 559/1112] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:09:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:09:16.699922 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc0e46-f105-4020-91c8-a311732ba405 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:16.801081 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc0e46-f105-4020-91c8-a311732ba405 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:16.828094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:16.834804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aeb2f5-70cd-4bc2-abdc-801b13b04a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:16.835616 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aeb2f5-70cd-4bc2-abdc-801b13b04a47 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:16.835903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aeb2f5-70cd-4bc2-abdc-801b13b04a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:16.836179 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aeb2f5-70cd-4bc2-abdc-801b13b04a47 None admin] PATH_INFO: `/v3/roles/6d02db1fe03b447c9e519f4f2aa16574` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:16.837655 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aeb2f5-70cd-4bc2-abdc-801b13b04a47 None admin] RBAC: Authorizing `identity:get_role(role_id=6d02db1fe03b447c9e519f4f2aa16574)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:16.839019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aeb2f5-70cd-4bc2-abdc-801b13b04a47 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:16.840431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 554/1113] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:09:16 2022] GET /identity/v3/roles/6d02db1fe03b447c9e519f4f2aa16574 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:16.846222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c6ca1-6a19-4285-b090-538721843f76 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:16.944395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c6ca1-6a19-4285-b090-538721843f76 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:16.978886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c395c-8d07-4a1d-91e9-ff38f0b056a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:16.979769 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c395c-8d07-4a1d-91e9-ff38f0b056a4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:16.980046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c395c-8d07-4a1d-91e9-ff38f0b056a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:16.980311 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c395c-8d07-4a1d-91e9-ff38f0b056a4 None admin] PATH_INFO: `/v3/users/8621db39d0b948119b23b6899dfdedb1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:16.992902 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c395c-8d07-4a1d-91e9-ff38f0b056a4 None admin] RBAC: Authorizing `identity:get_user(user_id=8621db39d0b948119b23b6899dfdedb1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:16.994670 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c395c-8d07-4a1d-91e9-ff38f0b056a4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:16.996344 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 560/1114] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:09:16 2022] GET /identity/v3/users/8621db39d0b948119b23b6899dfdedb1 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:17.002085 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aeb2f5-70cd-4bc2-abdc-801b13b04a47 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:17.099863 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aeb2f5-70cd-4bc2-abdc-801b13b04a47 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:17.133149 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad044f-80eb-4130-a779-ed25e3a012e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:17.133783 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad044f-80eb-4130-a779-ed25e3a012e5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:17.134063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad044f-80eb-4130-a779-ed25e3a012e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:17.134332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad044f-80eb-4130-a779-ed25e3a012e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:17.197149 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:17.201287 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ad044f-80eb-4130-a779-ed25e3a012e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:17.202730 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ad044f-80eb-4130-a779-ed25e3a012e5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:17.296292 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 555/1115] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:09:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:09:17.302505 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c395c-8d07-4a1d-91e9-ff38f0b056a4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:17.318074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:17.399887 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c395c-8d07-4a1d-91e9-ff38f0b056a4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:17.433519 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883850b-53de-4b98-9f99-60e2f6b35deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:17.434363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883850b-53de-4b98-9f99-60e2f6b35deb None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:17.434648 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883850b-53de-4b98-9f99-60e2f6b35deb None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:17.434923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883850b-53de-4b98-9f99-60e2f6b35deb None admin] PATH_INFO: `/v3/projects/288242db145547d480bf1522f1d0440f` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:17.436106 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7883850b-53de-4b98-9f99-60e2f6b35deb None admin] RBAC: Authorizing `identity:get_project(project_id=288242db145547d480bf1522f1d0440f)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:17.437525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7883850b-53de-4b98-9f99-60e2f6b35deb None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:17.439242 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 561/1116] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:09:17 2022] GET /identity/v3/projects/288242db145547d480bf1522f1d0440f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:17.446292 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad044f-80eb-4130-a779-ed25e3a012e5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:17.538292 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:17.542530 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad044f-80eb-4130-a779-ed25e3a012e5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:17.576535 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65530445-79d6-4ba8-920e-7d822a8ef721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:17.577180 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65530445-79d6-4ba8-920e-7d822a8ef721 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:17.577460 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65530445-79d6-4ba8-920e-7d822a8ef721 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:17.577731 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65530445-79d6-4ba8-920e-7d822a8ef721 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:17.578555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65530445-79d6-4ba8-920e-7d822a8ef721 None admin] RBAC: Adding query filter params (role.id=6d02db1fe03b447c9e519f4f2aa16574, scope.project.id=288242db145547d480bf1522f1d0440f, user.id=8621db39d0b948119b23b6899dfdedb1) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:17.578861 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65530445-79d6-4ba8-920e-7d822a8ef721 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:17.580301 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65530445-79d6-4ba8-920e-7d822a8ef721 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:17.585849 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 556/1117] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:09:17 2022] GET /identity/v3/role_assignments?user.id=8621db39d0b948119b23b6899dfdedb1&scope.project.id=288242db145547d480bf1522f1d0440f&role.id=6d02db1fe03b447c9e519f4f2aa16574 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:19.819732 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3253b-e147-4330-9c0a-29e0a060c2dd None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:19.820265 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3253b-e147-4330-9c0a-29e0a060c2dd None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:19.820265 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3253b-e147-4330-9c0a-29e0a060c2dd None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:19.821730 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 562/1118] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:09:19.826896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ef94b-cc43-40bf-9e62-c4d376b51c8e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:19.827352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ef94b-cc43-40bf-9e62-c4d376b51c8e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:19.827469 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ef94b-cc43-40bf-9e62-c4d376b51c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:19.844265 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-584ef94b-cc43-40bf-9e62-c4d376b51c8e None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:09:19.845431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-584ef94b-cc43-40bf-9e62-c4d376b51c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:09:19.909876 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 557/1119] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:09:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:09:19.933146 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3253b-e147-4330-9c0a-29e0a060c2dd None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:20.013159 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:20.032187 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3253b-e147-4330-9c0a-29e0a060c2dd None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:20.065162 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a605ae-3cf8-4e3b-8e8d-fdde08797099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:20.066004 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a605ae-3cf8-4e3b-8e8d-fdde08797099 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:20.066924 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a605ae-3cf8-4e3b-8e8d-fdde08797099 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:20.067195 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a605ae-3cf8-4e3b-8e8d-fdde08797099 None admin] PATH_INFO: `/v3/roles/6d02db1fe03b447c9e519f4f2aa16574` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:20.068692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a605ae-3cf8-4e3b-8e8d-fdde08797099 None admin] RBAC: Authorizing `identity:get_role(role_id=6d02db1fe03b447c9e519f4f2aa16574)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:20.070110 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a605ae-3cf8-4e3b-8e8d-fdde08797099 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:20.071460 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 563/1120] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:09:19 2022] GET /identity/v3/roles/6d02db1fe03b447c9e519f4f2aa16574 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:20.077349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ef94b-cc43-40bf-9e62-c4d376b51c8e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:20.166722 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:20.176123 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ef94b-cc43-40bf-9e62-c4d376b51c8e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:20.210567 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4df8f-6217-4b9f-b67b-a43f5b828441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:20.211475 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4df8f-6217-4b9f-b67b-a43f5b828441 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:20.211761 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4df8f-6217-4b9f-b67b-a43f5b828441 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:20.212037 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4df8f-6217-4b9f-b67b-a43f5b828441 None admin] PATH_INFO: `/v3/users/8621db39d0b948119b23b6899dfdedb1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:20.213227 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4df8f-6217-4b9f-b67b-a43f5b828441 None admin] RBAC: Authorizing `identity:get_user(user_id=8621db39d0b948119b23b6899dfdedb1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:20.214660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4df8f-6217-4b9f-b67b-a43f5b828441 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:20.216323 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 558/1121] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:09:20 2022] GET /identity/v3/users/8621db39d0b948119b23b6899dfdedb1 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:20.222184 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a605ae-3cf8-4e3b-8e8d-fdde08797099 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:20.320367 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a605ae-3cf8-4e3b-8e8d-fdde08797099 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:20.352733 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b83f15-76ea-438a-89f2-201f45800a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:20.353568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b83f15-76ea-438a-89f2-201f45800a2b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:20.353843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b83f15-76ea-438a-89f2-201f45800a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:20.354109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b83f15-76ea-438a-89f2-201f45800a2b None admin] PATH_INFO: `/v3/projects/288242db145547d480bf1522f1d0440f` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:20.355266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b83f15-76ea-438a-89f2-201f45800a2b None admin] RBAC: Authorizing `identity:get_project(project_id=288242db145547d480bf1522f1d0440f)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:20.356787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b83f15-76ea-438a-89f2-201f45800a2b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:20.358447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 564/1122] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:09:20 2022] GET /identity/v3/projects/288242db145547d480bf1522f1d0440f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:20.364358 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4df8f-6217-4b9f-b67b-a43f5b828441 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:20.460402 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4df8f-6217-4b9f-b67b-a43f5b828441 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:20.494912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e8a24-3564-4775-8a41-3399c3d8b823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:20.495824 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e8a24-3564-4775-8a41-3399c3d8b823 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:20.496107 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e8a24-3564-4775-8a41-3399c3d8b823 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:20.496383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e8a24-3564-4775-8a41-3399c3d8b823 None admin] PATH_INFO: `/v3/projects/288242db145547d480bf1522f1d0440f/users/8621db39d0b948119b23b6899dfdedb1/roles/6d02db1fe03b447c9e519f4f2aa16574` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:20.498549 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e8a24-3564-4775-8a41-3399c3d8b823 None admin] RBAC: Authorizing `identity:create_grant(project_id=288242db145547d480bf1522f1d0440f, user_id=8621db39d0b948119b23b6899dfdedb1, role_id=6d02db1fe03b447c9e519f4f2aa16574)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:20.499984 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e8a24-3564-4775-8a41-3399c3d8b823 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:20.512727 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 559/1123] 10.222.0.81 () {58 vars in 1439 bytes} [Tue Jun 7 14:09:20 2022] PUT /identity/v3/projects/288242db145547d480bf1522f1d0440f/users/8621db39d0b948119b23b6899dfdedb1/roles/6d02db1fe03b447c9e519f4f2aa16574 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:09:22.729433 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9354fac-fdcc-4dec-bd17-c984eef35786 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:22.729962 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9354fac-fdcc-4dec-bd17-c984eef35786 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:22.729962 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9354fac-fdcc-4dec-bd17-c984eef35786 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:22.731352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 565/1124] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:09:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:09:22.736616 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c848f8-f64c-4f4b-a907-d7c6d71f19ee None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:22.737040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c848f8-f64c-4f4b-a907-d7c6d71f19ee None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:22.737170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c848f8-f64c-4f4b-a907-d7c6d71f19ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:22.755155 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-48c848f8-f64c-4f4b-a907-d7c6d71f19ee None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:09:22.756349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-48c848f8-f64c-4f4b-a907-d7c6d71f19ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:09:22.769935 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:22.821184 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 560/1125] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:09:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:09:22.844934 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9354fac-fdcc-4dec-bd17-c984eef35786 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:22.864556 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:22.945549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9354fac-fdcc-4dec-bd17-c984eef35786 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:22.977301 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c6d8f-af22-415d-ae08-827ac362aff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:22.978134 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c6d8f-af22-415d-ae08-827ac362aff9 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:22.978411 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c6d8f-af22-415d-ae08-827ac362aff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:22.978681 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c6d8f-af22-415d-ae08-827ac362aff9 None admin] PATH_INFO: `/v3/roles/6d02db1fe03b447c9e519f4f2aa16574` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:22.980197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c6d8f-af22-415d-ae08-827ac362aff9 None admin] RBAC: Authorizing `identity:get_role(role_id=6d02db1fe03b447c9e519f4f2aa16574)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:22.981490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c6d8f-af22-415d-ae08-827ac362aff9 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:22.982940 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 566/1126] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:09:22 2022] GET /identity/v3/roles/6d02db1fe03b447c9e519f4f2aa16574 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:22.989236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c848f8-f64c-4f4b-a907-d7c6d71f19ee None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:23.086066 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c848f8-f64c-4f4b-a907-d7c6d71f19ee None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:23.120217 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3413e5e-7e51-4014-9453-3f7f5229f82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:23.121084 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3413e5e-7e51-4014-9453-3f7f5229f82f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:23.121367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3413e5e-7e51-4014-9453-3f7f5229f82f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:23.121642 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3413e5e-7e51-4014-9453-3f7f5229f82f None admin] PATH_INFO: `/v3/users/8621db39d0b948119b23b6899dfdedb1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:23.122821 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3413e5e-7e51-4014-9453-3f7f5229f82f None admin] RBAC: Authorizing `identity:get_user(user_id=8621db39d0b948119b23b6899dfdedb1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:23.124272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3413e5e-7e51-4014-9453-3f7f5229f82f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:23.125888 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 561/1127] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:09:22 2022] GET /identity/v3/users/8621db39d0b948119b23b6899dfdedb1 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:23.131939 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c6d8f-af22-415d-ae08-827ac362aff9 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:23.216459 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:23.231663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c6d8f-af22-415d-ae08-827ac362aff9 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:23.264408 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f866d4-cd4e-4580-89ed-e9c3d524cb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:23.265055 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f866d4-cd4e-4580-89ed-e9c3d524cb12 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:23.265338 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f866d4-cd4e-4580-89ed-e9c3d524cb12 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:23.265616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f866d4-cd4e-4580-89ed-e9c3d524cb12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:23.329690 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f866d4-cd4e-4580-89ed-e9c3d524cb12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:23.331117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f866d4-cd4e-4580-89ed-e9c3d524cb12 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:23.416380 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:23.431968 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 567/1128] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:09:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:09:23.438064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3413e5e-7e51-4014-9453-3f7f5229f82f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:23.457186 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:23.549293 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3413e5e-7e51-4014-9453-3f7f5229f82f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:23.586823 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32484fa5-eff3-4c5d-b625-43815469bce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:23.587722 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32484fa5-eff3-4c5d-b625-43815469bce8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:23.588005 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32484fa5-eff3-4c5d-b625-43815469bce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:23.588282 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32484fa5-eff3-4c5d-b625-43815469bce8 None admin] PATH_INFO: `/v3/projects/288242db145547d480bf1522f1d0440f` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:23.589468 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32484fa5-eff3-4c5d-b625-43815469bce8 None admin] RBAC: Authorizing `identity:get_project(project_id=288242db145547d480bf1522f1d0440f)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:23.591546 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32484fa5-eff3-4c5d-b625-43815469bce8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:23.593270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 562/1129] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:09:23 2022] GET /identity/v3/projects/288242db145547d480bf1522f1d0440f => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:23.600345 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f866d4-cd4e-4580-89ed-e9c3d524cb12 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:23.714612 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f866d4-cd4e-4580-89ed-e9c3d524cb12 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:23.753767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc1ccc-bade-4c80-b2d4-e0665aa4be50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:23.754514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc1ccc-bade-4c80-b2d4-e0665aa4be50 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:23.754691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc1ccc-bade-4c80-b2d4-e0665aa4be50 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:23.754965 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc1ccc-bade-4c80-b2d4-e0665aa4be50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:23.755821 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc1ccc-bade-4c80-b2d4-e0665aa4be50 None admin] RBAC: Adding query filter params (role.id=6d02db1fe03b447c9e519f4f2aa16574, scope.project.id=288242db145547d480bf1522f1d0440f, user.id=8621db39d0b948119b23b6899dfdedb1) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:23.756132 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc1ccc-bade-4c80-b2d4-e0665aa4be50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:23.757970 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc1ccc-bade-4c80-b2d4-e0665aa4be50 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:23.764805 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 568/1130] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:09:23 2022] GET /identity/v3/role_assignments?user.id=8621db39d0b948119b23b6899dfdedb1&scope.project.id=288242db145547d480bf1522f1d0440f&role.id=6d02db1fe03b447c9e519f4f2aa16574 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:26.038933 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ef17f-041e-407c-bf22-979e8582fc9c None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:26.039518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ef17f-041e-407c-bf22-979e8582fc9c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:26.039518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ef17f-041e-407c-bf22-979e8582fc9c None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:26.040897 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 563/1131] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:09:26.046178 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4b0d4-27d2-4ca4-bf9c-0f702895e8b2 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:26.046561 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4b0d4-27d2-4ca4-bf9c-0f702895e8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:26.046735 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4b0d4-27d2-4ca4-bf9c-0f702895e8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:26.063289 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-58a4b0d4-27d2-4ca4-bf9c-0f702895e8b2 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:09:26.064475 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-58a4b0d4-27d2-4ca4-bf9c-0f702895e8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:09:26.109845 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:26.131051 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 569/1132] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:09:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:09:26.154994 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ef17f-041e-407c-bf22-979e8582fc9c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:26.239335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:26.255109 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ef17f-041e-407c-bf22-979e8582fc9c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:26.287692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d85b8a-db9a-4363-80fd-6a1c0e3e4070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:26.288473 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d85b8a-db9a-4363-80fd-6a1c0e3e4070 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:26.288757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d85b8a-db9a-4363-80fd-6a1c0e3e4070 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:26.289079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d85b8a-db9a-4363-80fd-6a1c0e3e4070 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:26.290197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d85b8a-db9a-4363-80fd-6a1c0e3e4070 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:26.291654 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d85b8a-db9a-4363-80fd-6a1c0e3e4070 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:26.293262 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 564/1133] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:09:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:26.299145 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4b0d4-27d2-4ca4-bf9c-0f702895e8b2 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:26.395172 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4b0d4-27d2-4ca4-bf9c-0f702895e8b2 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:26.428000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f4938-1cb1-4b7a-80d3-b07dfc724311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:26.428657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f4938-1cb1-4b7a-80d3-b07dfc724311 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:26.428940 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f4938-1cb1-4b7a-80d3-b07dfc724311 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:26.429215 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f4938-1cb1-4b7a-80d3-b07dfc724311 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:26.429924 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f4938-1cb1-4b7a-80d3-b07dfc724311 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:26.431399 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f4938-1cb1-4b7a-80d3-b07dfc724311 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:26.455879 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 570/1134] 10.222.0.81 () {60 vars in 1141 bytes} [Tue Jun 7 14:09:26 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:09:28.752811 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012cbd4-5105-4d86-9422-29af506d7daa None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:28.753943 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012cbd4-5105-4d86-9422-29af506d7daa None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:28.753943 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012cbd4-5105-4d86-9422-29af506d7daa None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:28.755293 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 565/1135] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:09:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:09:28.760535 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00324f9e-b6ec-464f-bb0f-e4ef8f5f9381 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:28.760940 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00324f9e-b6ec-464f-bb0f-e4ef8f5f9381 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:28.761094 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00324f9e-b6ec-464f-bb0f-e4ef8f5f9381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:28.779340 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-00324f9e-b6ec-464f-bb0f-e4ef8f5f9381 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:09:28.780522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-00324f9e-b6ec-464f-bb0f-e4ef8f5f9381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:09:28.805942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:28.845946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 571/1136] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:09:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:09:28.871846 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012cbd4-5105-4d86-9422-29af506d7daa None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:28.970211 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012cbd4-5105-4d86-9422-29af506d7daa None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:29.004137 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673fa3fb-e2ba-4917-8714-e0be3a539758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:29.005017 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fa3fb-e2ba-4917-8714-e0be3a539758 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:29.005296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fa3fb-e2ba-4917-8714-e0be3a539758 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:29.005565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fa3fb-e2ba-4917-8714-e0be3a539758 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:29.006731 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673fa3fb-e2ba-4917-8714-e0be3a539758 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:29.008172 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673fa3fb-e2ba-4917-8714-e0be3a539758 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:29.009766 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 566/1137] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:09:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:29.015463 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00324f9e-b6ec-464f-bb0f-e4ef8f5f9381 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:29.110959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00324f9e-b6ec-464f-bb0f-e4ef8f5f9381 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:29.144334 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191e033-794e-439d-b850-ee69accef9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:29.144988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191e033-794e-439d-b850-ee69accef9fa None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:29.145268 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191e033-794e-439d-b850-ee69accef9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:29.145540 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191e033-794e-439d-b850-ee69accef9fa None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:29.146251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191e033-794e-439d-b850-ee69accef9fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:29.147699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191e033-794e-439d-b850-ee69accef9fa None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:29.168808 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 572/1138] 10.222.0.81 () {60 vars in 1132 bytes} [Tue Jun 7 14:09:29 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:09:31.444998 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db7dc1-714d-4c93-8938-5bdb26481348 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:31.445514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db7dc1-714d-4c93-8938-5bdb26481348 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:31.445514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db7dc1-714d-4c93-8938-5bdb26481348 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:31.446931 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 567/1139] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:09:31.452347 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d2ae8-58e0-4575-925d-b35c087b5a1d None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:31.452743 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d2ae8-58e0-4575-925d-b35c087b5a1d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:31.452902 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d2ae8-58e0-4575-925d-b35c087b5a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:31.469902 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-844d2ae8-58e0-4575-925d-b35c087b5a1d None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:09:31.471076 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-844d2ae8-58e0-4575-925d-b35c087b5a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:09:31.484214 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:31.538470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 573/1140] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:09:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:09:31.563777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db7dc1-714d-4c93-8938-5bdb26481348 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:31.587521 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:31.663761 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db7dc1-714d-4c93-8938-5bdb26481348 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:31.696099 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e326cc-d231-4c41-adf8-96d67d817d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:31.696814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e326cc-d231-4c41-adf8-96d67d817d2a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:31.697095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e326cc-d231-4c41-adf8-96d67d817d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:31.697364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e326cc-d231-4c41-adf8-96d67d817d2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:31.707885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e326cc-d231-4c41-adf8-96d67d817d2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:09:31.715373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-78e326cc-d231-4c41-adf8-96d67d817d2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:09:31.715373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:31.715373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:31.715373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:31.715373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:31.715373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:31.715373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:31.715373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:31.715373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:31.715373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:31.715373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:31.715373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:31.715373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:09:31.715373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:09:31.715373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:09:31.715373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:09:31.716447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:09:31.716447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:09:31.716447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:09:31.716447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:09:31.716447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:31.716447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:31.716447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:31.716447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:31.716447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:31.716447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:31.716447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:31.716447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:31.716447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:31.716447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:31.716447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:31.716447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:31.717345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:31.717345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:31.717345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:31.717345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:31.717345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:09:31.717345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:09:31.717345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:09:31.717345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:09:31.717345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:09:31.717345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:09:31.717345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:09:31.717345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:09:31.717345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 568/1141] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:09:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:09:31.723342 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d2ae8-58e0-4575-925d-b35c087b5a1d None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:31.816847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d2ae8-58e0-4575-925d-b35c087b5a1d None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:31.850349 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199e964-5932-41a2-b44f-281a0edc1a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:31.851010 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199e964-5932-41a2-b44f-281a0edc1a46 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:31.851316 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199e964-5932-41a2-b44f-281a0edc1a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:31.851596 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199e964-5932-41a2-b44f-281a0edc1a46 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:31.852333 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199e964-5932-41a2-b44f-281a0edc1a46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:31.852638 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199e964-5932-41a2-b44f-281a0edc1a46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:31.853984 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199e964-5932-41a2-b44f-281a0edc1a46 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:31.861739 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 574/1142] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:09:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:31.867710 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e326cc-d231-4c41-adf8-96d67d817d2a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:31.948440 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:31.966723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e326cc-d231-4c41-adf8-96d67d817d2a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:32.000104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2abc015-1105-4bbc-92da-f6f826824dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:32.000964 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abc015-1105-4bbc-92da-f6f826824dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:32.001245 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abc015-1105-4bbc-92da-f6f826824dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:32.001519 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abc015-1105-4bbc-92da-f6f826824dc7 None admin] PATH_INFO: `/v3/users/e47b8b5465a04399a008f21b08a9dacd` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:32.017690 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2abc015-1105-4bbc-92da-f6f826824dc7 None admin] RBAC: Authorizing `identity:get_user(user_id=e47b8b5465a04399a008f21b08a9dacd)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:32.020530 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2abc015-1105-4bbc-92da-f6f826824dc7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:32.022205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 569/1143] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:09:31 2022] GET /identity/v3/users/e47b8b5465a04399a008f21b08a9dacd => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:32.027978 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199e964-5932-41a2-b44f-281a0edc1a46 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:32.040984 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:32.126091 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199e964-5932-41a2-b44f-281a0edc1a46 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:32.159064 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0098b-8bc5-4ca3-b999-2d4901911610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:32.159859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0098b-8bc5-4ca3-b999-2d4901911610 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:32.160227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0098b-8bc5-4ca3-b999-2d4901911610 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:32.160503 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0098b-8bc5-4ca3-b999-2d4901911610 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:32.225420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e0098b-8bc5-4ca3-b999-2d4901911610 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:32.226867 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e0098b-8bc5-4ca3-b999-2d4901911610 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:32.237645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:32.323508 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 575/1144] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:09:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:09:32.329633 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2abc015-1105-4bbc-92da-f6f826824dc7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:32.424832 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2abc015-1105-4bbc-92da-f6f826824dc7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:32.457602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce430c93-ee45-48f1-af1e-38ca6583564a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:32.458446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce430c93-ee45-48f1-af1e-38ca6583564a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:32.458724 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce430c93-ee45-48f1-af1e-38ca6583564a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:32.458996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce430c93-ee45-48f1-af1e-38ca6583564a None admin] PATH_INFO: `/v3/projects/48ca44acd5ff4c048d35477eccf214a7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:32.460171 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce430c93-ee45-48f1-af1e-38ca6583564a None admin] RBAC: Authorizing `identity:get_project(project_id=48ca44acd5ff4c048d35477eccf214a7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:32.461592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce430c93-ee45-48f1-af1e-38ca6583564a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:32.463275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 570/1145] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:09:32 2022] GET /identity/v3/projects/48ca44acd5ff4c048d35477eccf214a7 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:32.469244 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0098b-8bc5-4ca3-b999-2d4901911610 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:32.565410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0098b-8bc5-4ca3-b999-2d4901911610 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:32.576307 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:32.599151 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebcecf-ac18-4e9e-bcb2-e37d1ea33334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:32.599834 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebcecf-ac18-4e9e-bcb2-e37d1ea33334 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:32.600119 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebcecf-ac18-4e9e-bcb2-e37d1ea33334 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:32.600397 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebcecf-ac18-4e9e-bcb2-e37d1ea33334 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:32.601217 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeebcecf-ac18-4e9e-bcb2-e37d1ea33334 None admin] RBAC: Adding query filter params (role.id=82bbe3651b9b4f378fd150d93357b2b1, scope.project.id=48ca44acd5ff4c048d35477eccf214a7, user.id=e47b8b5465a04399a008f21b08a9dacd) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:32.602287 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeebcecf-ac18-4e9e-bcb2-e37d1ea33334 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:32.602949 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeebcecf-ac18-4e9e-bcb2-e37d1ea33334 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:32.608340 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 576/1146] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:09:32 2022] GET /identity/v3/role_assignments?user.id=e47b8b5465a04399a008f21b08a9dacd&scope.project.id=48ca44acd5ff4c048d35477eccf214a7&role.id=82bbe3651b9b4f378fd150d93357b2b1 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:34.823904 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d0a7b-633b-4ebf-ab9b-cd38a792ce07 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:34.824504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d0a7b-633b-4ebf-ab9b-cd38a792ce07 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:34.824504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d0a7b-633b-4ebf-ab9b-cd38a792ce07 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:34.825825 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 571/1147] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:09:34.831082 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e14656-73b6-4d75-b6b9-cfd1f653764f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:34.831546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e14656-73b6-4d75-b6b9-cfd1f653764f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:34.831688 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e14656-73b6-4d75-b6b9-cfd1f653764f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:34.848175 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-76e14656-73b6-4d75-b6b9-cfd1f653764f None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:09:34.849332 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-76e14656-73b6-4d75-b6b9-cfd1f653764f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:09:34.914353 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 577/1148] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:09:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:09:34.937189 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d0a7b-633b-4ebf-ab9b-cd38a792ce07 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:34.948566 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:35.036825 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d0a7b-633b-4ebf-ab9b-cd38a792ce07 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:35.069298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f291e82-2c19-4e59-be81-5271216a1844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:35.070073 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f291e82-2c19-4e59-be81-5271216a1844 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:35.070353 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f291e82-2c19-4e59-be81-5271216a1844 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:35.070623 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f291e82-2c19-4e59-be81-5271216a1844 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:35.080499 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f291e82-2c19-4e59-be81-5271216a1844 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:09:35.088257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-0f291e82-2c19-4e59-be81-5271216a1844 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:09:35.088257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:35.088257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:35.088257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:35.088257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:35.088257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:35.088257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:35.088257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:35.088257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:35.088257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:35.088257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:35.088257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:35.088257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:09:35.088257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:09:35.088257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:09:35.088257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:09:35.089383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:09:35.089383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:09:35.089383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:09:35.089383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:09:35.089383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:35.089383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:35.089383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:35.089383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:35.089383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:35.089383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:35.089383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:35.089383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:35.089383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:35.089383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:35.089383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:35.089383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:35.090289 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:35.090289 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:35.090289 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:35.090289 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:35.090289 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:09:35.090289 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:09:35.090289 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:09:35.090289 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:09:35.090289 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:09:35.090289 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:09:35.090289 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:09:35.090289 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:09:35.090289 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 572/1149] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:09:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:09:35.095442 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e14656-73b6-4d75-b6b9-cfd1f653764f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:35.194374 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e14656-73b6-4d75-b6b9-cfd1f653764f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:35.196720 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:35.227525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e29ad8-5559-45b3-8e4c-7bcf4f8327ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:35.228178 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e29ad8-5559-45b3-8e4c-7bcf4f8327ef None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:35.228458 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e29ad8-5559-45b3-8e4c-7bcf4f8327ef None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:35.228728 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e29ad8-5559-45b3-8e4c-7bcf4f8327ef None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:35.229459 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e29ad8-5559-45b3-8e4c-7bcf4f8327ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:35.229758 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e29ad8-5559-45b3-8e4c-7bcf4f8327ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:35.231102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e29ad8-5559-45b3-8e4c-7bcf4f8327ef None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:35.239785 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 578/1150] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:09:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:35.246039 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f291e82-2c19-4e59-be81-5271216a1844 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:35.310402 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:35.343962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f291e82-2c19-4e59-be81-5271216a1844 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:35.377296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a59be-1998-40d5-9c90-35e26362a773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:35.378243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a59be-1998-40d5-9c90-35e26362a773 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:35.378726 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a59be-1998-40d5-9c90-35e26362a773 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:35.379001 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a59be-1998-40d5-9c90-35e26362a773 None admin] PATH_INFO: `/v3/users/e47b8b5465a04399a008f21b08a9dacd` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:35.380194 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a59be-1998-40d5-9c90-35e26362a773 None admin] RBAC: Authorizing `identity:get_user(user_id=e47b8b5465a04399a008f21b08a9dacd)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:35.381605 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a59be-1998-40d5-9c90-35e26362a773 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:35.383232 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 573/1151] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:09:35 2022] GET /identity/v3/users/e47b8b5465a04399a008f21b08a9dacd => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:35.389042 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e29ad8-5559-45b3-8e4c-7bcf4f8327ef None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:35.486191 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e29ad8-5559-45b3-8e4c-7bcf4f8327ef None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:35.518133 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13daef-cc49-45de-878a-9fc05719532e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:35.519534 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13daef-cc49-45de-878a-9fc05719532e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:35.519963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13daef-cc49-45de-878a-9fc05719532e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:35.520239 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13daef-cc49-45de-878a-9fc05719532e None admin] PATH_INFO: `/v3/projects/48ca44acd5ff4c048d35477eccf214a7` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:35.521399 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e13daef-cc49-45de-878a-9fc05719532e None admin] RBAC: Authorizing `identity:get_project(project_id=48ca44acd5ff4c048d35477eccf214a7)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:35.522830 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e13daef-cc49-45de-878a-9fc05719532e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:35.524536 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 579/1152] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:09:35 2022] GET /identity/v3/projects/48ca44acd5ff4c048d35477eccf214a7 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:35.530248 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a59be-1998-40d5-9c90-35e26362a773 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:35.628278 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a59be-1998-40d5-9c90-35e26362a773 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:35.661578 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a0cc3-8fa7-402f-8f17-3932892083d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:35.662454 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a0cc3-8fa7-402f-8f17-3932892083d7 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:35.662736 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a0cc3-8fa7-402f-8f17-3932892083d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:35.663009 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a0cc3-8fa7-402f-8f17-3932892083d7 None admin] PATH_INFO: `/v3/projects/48ca44acd5ff4c048d35477eccf214a7/users/e47b8b5465a04399a008f21b08a9dacd/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:35.664982 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a0cc3-8fa7-402f-8f17-3932892083d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=48ca44acd5ff4c048d35477eccf214a7, user_id=e47b8b5465a04399a008f21b08a9dacd, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:35.666397 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a0cc3-8fa7-402f-8f17-3932892083d7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:35.669344 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:35.681702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 574/1153] 10.222.0.81 () {58 vars in 1439 bytes} [Tue Jun 7 14:09:35 2022] PUT /identity/v3/projects/48ca44acd5ff4c048d35477eccf214a7/users/e47b8b5465a04399a008f21b08a9dacd/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:09:37.885429 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f877c-4e3d-400b-9615-adadbc8699b0 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:37.885962 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f877c-4e3d-400b-9615-adadbc8699b0 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:37.885962 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f877c-4e3d-400b-9615-adadbc8699b0 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:37.887384 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 580/1154] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:09:37.892664 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145d42f-25b8-44fe-b958-2ca5d731d7e9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:37.893083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145d42f-25b8-44fe-b958-2ca5d731d7e9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:37.893305 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145d42f-25b8-44fe-b958-2ca5d731d7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:37.911908 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3145d42f-25b8-44fe-b958-2ca5d731d7e9 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:09:37.913095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3145d42f-25b8-44fe-b958-2ca5d731d7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:09:37.976818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 575/1155] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:09:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:09:38.000097 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f877c-4e3d-400b-9615-adadbc8699b0 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:38.035818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:38.100653 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f877c-4e3d-400b-9615-adadbc8699b0 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:38.133696 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb59f7-7522-4c9f-b3d3-48e29320746d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:38.134474 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb59f7-7522-4c9f-b3d3-48e29320746d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:38.134753 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb59f7-7522-4c9f-b3d3-48e29320746d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:38.135024 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb59f7-7522-4c9f-b3d3-48e29320746d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:38.145542 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb59f7-7522-4c9f-b3d3-48e29320746d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:09:38.153046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-68fb59f7-7522-4c9f-b3d3-48e29320746d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:09:38.153046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:38.153046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:38.153046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:38.153046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:38.153046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:38.153046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:38.153046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:38.153046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:38.153046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:38.153046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:38.153046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:38.153046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:09:38.153046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:09:38.153046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:09:38.153046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:09:38.154045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:09:38.154045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:09:38.154045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:09:38.154045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:09:38.154045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:38.154045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:38.154045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:38.154045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:38.154045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:38.154045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:38.154045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:38.154045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:38.154045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:38.154045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:38.154045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:38.154045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:38.154909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:38.154909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:38.154909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:38.154909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:38.154909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:09:38.154909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:09:38.154909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:09:38.154909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:09:38.154909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:09:38.154909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:09:38.154909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:09:38.154909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:09:38.154909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 581/1156] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:09:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:09:38.160284 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3145d42f-25b8-44fe-b958-2ca5d731d7e9 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:38.260531 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3145d42f-25b8-44fe-b958-2ca5d731d7e9 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:38.285836 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:38.294474 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c499d-f7d9-41b0-9203-0cdb226a56bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:38.295139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c499d-f7d9-41b0-9203-0cdb226a56bc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:38.295453 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c499d-f7d9-41b0-9203-0cdb226a56bc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:38.295724 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c499d-f7d9-41b0-9203-0cdb226a56bc None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:38.296462 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c499d-f7d9-41b0-9203-0cdb226a56bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:38.296764 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c499d-f7d9-41b0-9203-0cdb226a56bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:38.298119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c499d-f7d9-41b0-9203-0cdb226a56bc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:38.305939 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 576/1157] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:09:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:38.311717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb59f7-7522-4c9f-b3d3-48e29320746d None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:38.404033 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:38.410843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb59f7-7522-4c9f-b3d3-48e29320746d None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:38.443453 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3a15e-f52c-4412-95ec-8f0f71d031ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:38.444312 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3a15e-f52c-4412-95ec-8f0f71d031ba None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:38.444590 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3a15e-f52c-4412-95ec-8f0f71d031ba None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:38.444930 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3a15e-f52c-4412-95ec-8f0f71d031ba None admin] PATH_INFO: `/v3/users/e47b8b5465a04399a008f21b08a9dacd` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:38.446120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3a15e-f52c-4412-95ec-8f0f71d031ba None admin] RBAC: Authorizing `identity:get_user(user_id=e47b8b5465a04399a008f21b08a9dacd)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:38.447568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3a15e-f52c-4412-95ec-8f0f71d031ba None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:38.449166 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 582/1158] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:09:38 2022] GET /identity/v3/users/e47b8b5465a04399a008f21b08a9dacd => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:38.454884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c499d-f7d9-41b0-9203-0cdb226a56bc None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:38.551104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c499d-f7d9-41b0-9203-0cdb226a56bc None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:38.583424 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ad52f-20bd-4820-bf19-26b3981ed218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:38.584066 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ad52f-20bd-4820-bf19-26b3981ed218 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:38.584351 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ad52f-20bd-4820-bf19-26b3981ed218 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:38.584626 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ad52f-20bd-4820-bf19-26b3981ed218 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:38.637699 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:38.650377 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ad52f-20bd-4820-bf19-26b3981ed218 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:38.651842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ad52f-20bd-4820-bf19-26b3981ed218 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:38.746374 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 577/1159] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:09:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:09:38.752199 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3a15e-f52c-4412-95ec-8f0f71d031ba None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:38.847435 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3a15e-f52c-4412-95ec-8f0f71d031ba None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:38.879476 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0b478-d23e-4341-85c7-a0d407d40c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:38.880318 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0b478-d23e-4341-85c7-a0d407d40c79 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:38.880597 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0b478-d23e-4341-85c7-a0d407d40c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:38.880869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0b478-d23e-4341-85c7-a0d407d40c79 None admin] PATH_INFO: `/v3/projects/48ca44acd5ff4c048d35477eccf214a7` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:38.882024 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0b478-d23e-4341-85c7-a0d407d40c79 None admin] RBAC: Authorizing `identity:get_project(project_id=48ca44acd5ff4c048d35477eccf214a7)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:38.883460 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0b478-d23e-4341-85c7-a0d407d40c79 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:38.885135 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 583/1160] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:09:38 2022] GET /identity/v3/projects/48ca44acd5ff4c048d35477eccf214a7 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:38.891389 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ad52f-20bd-4820-bf19-26b3981ed218 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:38.976123 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:38.987929 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ad52f-20bd-4820-bf19-26b3981ed218 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:39.022027 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d7b96-6b14-411c-bf32-95dc32932423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:39.022605 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d7b96-6b14-411c-bf32-95dc32932423 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:39.022886 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d7b96-6b14-411c-bf32-95dc32932423 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:39.023159 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d7b96-6b14-411c-bf32-95dc32932423 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:39.024005 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d7b96-6b14-411c-bf32-95dc32932423 None admin] RBAC: Adding query filter params (role.id=82bbe3651b9b4f378fd150d93357b2b1, scope.project.id=48ca44acd5ff4c048d35477eccf214a7, user.id=e47b8b5465a04399a008f21b08a9dacd) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:39.024316 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d7b96-6b14-411c-bf32-95dc32932423 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:39.025729 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d7b96-6b14-411c-bf32-95dc32932423 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:39.031392 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 578/1161] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:09:38 2022] GET /identity/v3/role_assignments?user.id=e47b8b5465a04399a008f21b08a9dacd&scope.project.id=48ca44acd5ff4c048d35477eccf214a7&role.id=82bbe3651b9b4f378fd150d93357b2b1 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:41.291904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56645c-5127-4377-b69d-c8bdaf929a94 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:41.292452 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56645c-5127-4377-b69d-c8bdaf929a94 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:41.292452 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56645c-5127-4377-b69d-c8bdaf929a94 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:41.293821 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 584/1162] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:09:41.299159 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d396d3b-e385-4224-90da-b772a7995251 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:41.299629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d396d3b-e385-4224-90da-b772a7995251 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:41.300460 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d396d3b-e385-4224-90da-b772a7995251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:41.316781 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6d396d3b-e385-4224-90da-b772a7995251 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:09:41.317948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6d396d3b-e385-4224-90da-b772a7995251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:09:41.382329 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 579/1163] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:09:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:09:41.405882 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56645c-5127-4377-b69d-c8bdaf929a94 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:41.436449 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:41.506290 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56645c-5127-4377-b69d-c8bdaf929a94 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:41.539359 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f2fb4-1860-4a99-a306-f2266efd6080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:41.540019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f2fb4-1860-4a99-a306-f2266efd6080 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:41.540296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f2fb4-1860-4a99-a306-f2266efd6080 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:41.540565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f2fb4-1860-4a99-a306-f2266efd6080 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:41.604476 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f2fb4-1860-4a99-a306-f2266efd6080 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:41.605893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f2fb4-1860-4a99-a306-f2266efd6080 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:41.634638 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:41.702470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 585/1164] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:09:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:09:41.708764 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d396d3b-e385-4224-90da-b772a7995251 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:41.787476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:41.805377 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d396d3b-e385-4224-90da-b772a7995251 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:41.837380 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0239c-f79e-492f-aa52-ee757b6d0aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:41.838172 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0239c-f79e-492f-aa52-ee757b6d0aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:41.838455 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0239c-f79e-492f-aa52-ee757b6d0aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:41.838729 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0239c-f79e-492f-aa52-ee757b6d0aa5 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:41.845244 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0239c-f79e-492f-aa52-ee757b6d0aa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:41.847266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0239c-f79e-492f-aa52-ee757b6d0aa5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:41.861130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f7e0239c-f79e-492f-aa52-ee757b6d0aa5 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 07 14:09:41.861130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:41.861130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 07 14:09:41.861130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 07 14:09:41.861130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:09:41.861130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:09:41.861130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:09:41.861130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:09:41.861130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 07 14:09:41.861130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:09:41.861130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:09:41.861130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:09:41.861130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:41.861130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:41.861130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:41.861130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:41.861130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:41.862348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:41.862348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:41.862348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:41.862348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:41.862348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:41.862348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:41.862348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:09:41.862348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:09:41.862348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:09:41.862348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:09:41.862348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:41.862348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:41.862348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:41.862348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:41.862348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:41.862348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:41.864295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:41.864295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:41.864295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:41.864295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:41.864295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:41.864295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:41.864295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:41.864295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:41.864295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:41.864295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:41.864295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 07 14:09:41.864295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:09:41.864295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 07 14:09:41.864295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:09:41.864295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 580/1165] 10.222.0.81 () {56 vars in 1117 bytes} [Tue Jun 7 14:09:41 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:09:41.869356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f2fb4-1860-4a99-a306-f2266efd6080 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:41.964383 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f2fb4-1860-4a99-a306-f2266efd6080 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:41.993180 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:41.999301 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a811a8c-134a-4b55-831f-3b3fc19655ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:41.999962 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a811a8c-134a-4b55-831f-3b3fc19655ab None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:42.000238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a811a8c-134a-4b55-831f-3b3fc19655ab None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:42.000507 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a811a8c-134a-4b55-831f-3b3fc19655ab None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:42.001214 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a811a8c-134a-4b55-831f-3b3fc19655ab None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:42.001515 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a811a8c-134a-4b55-831f-3b3fc19655ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:42.002921 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a811a8c-134a-4b55-831f-3b3fc19655ab None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:42.010095 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 586/1166] 10.222.0.81 () {56 vars in 1115 bytes} [Tue Jun 7 14:09:41 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:44.256239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b54df0-8e84-4a2c-a3d0-d27f2449d32d None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:44.256870 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b54df0-8e84-4a2c-a3d0-d27f2449d32d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:44.256870 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b54df0-8e84-4a2c-a3d0-d27f2449d32d None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:44.258171 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 581/1167] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:09:44.263315 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ca2a5-7b81-40b4-b17e-8243721f2429 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:44.263692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ca2a5-7b81-40b4-b17e-8243721f2429 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:44.263931 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ca2a5-7b81-40b4-b17e-8243721f2429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:44.280759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-755ca2a5-7b81-40b4-b17e-8243721f2429 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:09:44.281924 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-755ca2a5-7b81-40b4-b17e-8243721f2429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:09:44.346347 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 587/1168] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:09:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:09:44.370404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b54df0-8e84-4a2c-a3d0-d27f2449d32d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:44.468749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b54df0-8e84-4a2c-a3d0-d27f2449d32d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:44.501288 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd261d-8b7b-4615-a4d9-6d9c2901ab0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:44.501939 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd261d-8b7b-4615-a4d9-6d9c2901ab0b None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:44.502223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd261d-8b7b-4615-a4d9-6d9c2901ab0b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:44.502500 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd261d-8b7b-4615-a4d9-6d9c2901ab0b None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:44.503266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dd261d-8b7b-4615-a4d9-6d9c2901ab0b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:44.504624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dd261d-8b7b-4615-a4d9-6d9c2901ab0b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:44.523002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 582/1169] 10.222.0.81 () {60 vars in 1138 bytes} [Tue Jun 7 14:09:44 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:09:46.794244 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524eea1-780e-4096-a36e-0dff12af5e59 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:46.794772 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524eea1-780e-4096-a36e-0dff12af5e59 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:46.794772 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524eea1-780e-4096-a36e-0dff12af5e59 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:46.796190 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 588/1170] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:09:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:09:46.801984 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ab649-5aba-4ec0-9643-7edb868d1187 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:46.802468 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ab649-5aba-4ec0-9643-7edb868d1187 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:46.802560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ab649-5aba-4ec0-9643-7edb868d1187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:46.807551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:46.821518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-cd3ab649-5aba-4ec0-9643-7edb868d1187 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:09:46.822697 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cd3ab649-5aba-4ec0-9643-7edb868d1187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:09:46.886918 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 583/1171] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:09:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:09:46.910586 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524eea1-780e-4096-a36e-0dff12af5e59 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:47.009063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524eea1-780e-4096-a36e-0dff12af5e59 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:47.031191 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:47.042797 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a686e55c-83c4-4658-9ed6-1f2f31cebede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:47.043683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686e55c-83c4-4658-9ed6-1f2f31cebede None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:47.043967 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686e55c-83c4-4658-9ed6-1f2f31cebede None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:47.044241 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686e55c-83c4-4658-9ed6-1f2f31cebede None admin] PATH_INFO: `/v3/domains/0025715373094ab2a31040af59a52d2f` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:47.055335 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a686e55c-83c4-4658-9ed6-1f2f31cebede None admin] RBAC: Authorizing `identity:get_domain(domain_id=0025715373094ab2a31040af59a52d2f)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:47.056713 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a686e55c-83c4-4658-9ed6-1f2f31cebede None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:47.058377 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 589/1172] 10.222.0.81 () {56 vars in 1183 bytes} [Tue Jun 7 14:09:46 2022] GET /identity/v3/domains/0025715373094ab2a31040af59a52d2f => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:47.064134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ab649-5aba-4ec0-9643-7edb868d1187 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:47.160200 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ab649-5aba-4ec0-9643-7edb868d1187 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:47.196610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:47.201939 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d08ff-c31b-49bb-8bae-e8e6a496c1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:47.202598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d08ff-c31b-49bb-8bae-e8e6a496c1b2 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:47.202878 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d08ff-c31b-49bb-8bae-e8e6a496c1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:47.203149 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d08ff-c31b-49bb-8bae-e8e6a496c1b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:47.203887 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d08ff-c31b-49bb-8bae-e8e6a496c1b2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:47.205308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d08ff-c31b-49bb-8bae-e8e6a496c1b2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:47.229724 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 584/1173] 10.222.0.81 () {60 vars in 1141 bytes} [Tue Jun 7 14:09:47 2022] POST /identity/v3/projects => generated 361 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:09:49.510142 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bea3d9-046f-48bc-a671-c0f950d7ed65 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:49.510662 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bea3d9-046f-48bc-a671-c0f950d7ed65 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:49.510786 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bea3d9-046f-48bc-a671-c0f950d7ed65 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:49.512088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 590/1174] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:09:49.517433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a11004-437c-4c33-a396-0d4185222f04 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:49.517808 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a11004-437c-4c33-a396-0d4185222f04 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:49.517997 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a11004-437c-4c33-a396-0d4185222f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:49.535004 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d7a11004-437c-4c33-a396-0d4185222f04 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:09:49.536218 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d7a11004-437c-4c33-a396-0d4185222f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:09:49.600838 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 585/1175] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:09:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:09:49.626741 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bea3d9-046f-48bc-a671-c0f950d7ed65 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:49.727190 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bea3d9-046f-48bc-a671-c0f950d7ed65 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:49.759821 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de5a91-e01b-4e1d-aa9d-588d4baf334b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:49.760731 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de5a91-e01b-4e1d-aa9d-588d4baf334b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:49.760967 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de5a91-e01b-4e1d-aa9d-588d4baf334b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:49.761246 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de5a91-e01b-4e1d-aa9d-588d4baf334b None admin] PATH_INFO: `/v3/domains/0025715373094ab2a31040af59a52d2f` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:49.762417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de5a91-e01b-4e1d-aa9d-588d4baf334b None admin] RBAC: Authorizing `identity:get_domain(domain_id=0025715373094ab2a31040af59a52d2f)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:49.763880 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de5a91-e01b-4e1d-aa9d-588d4baf334b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:49.765509 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 591/1176] 10.222.0.81 () {56 vars in 1183 bytes} [Tue Jun 7 14:09:49 2022] GET /identity/v3/domains/0025715373094ab2a31040af59a52d2f => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:49.771960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a11004-437c-4c33-a396-0d4185222f04 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:49.867177 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a11004-437c-4c33-a396-0d4185222f04 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:49.880976 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:49.902719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cf9d78-e4ce-4455-9073-c8387071f818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:49.903419 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf9d78-e4ce-4455-9073-c8387071f818 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:49.903707 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf9d78-e4ce-4455-9073-c8387071f818 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:49.903982 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf9d78-e4ce-4455-9073-c8387071f818 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:49.904704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cf9d78-e4ce-4455-9073-c8387071f818 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:49.906148 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cf9d78-e4ce-4455-9073-c8387071f818 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:49.925486 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 586/1177] 10.222.0.81 () {60 vars in 1132 bytes} [Tue Jun 7 14:09:49 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:09:52.209042 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8976e5-fc4d-4710-a487-11cbfd7726cc None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:52.209560 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8976e5-fc4d-4710-a487-11cbfd7726cc None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:52.209660 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8976e5-fc4d-4710-a487-11cbfd7726cc None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:52.211004 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 592/1178] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:09:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:09:52.216384 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9fd0d0-991c-4a20-a748-ed8c29a4142b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:52.216789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9fd0d0-991c-4a20-a748-ed8c29a4142b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:52.216931 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9fd0d0-991c-4a20-a748-ed8c29a4142b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:52.234010 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2c9fd0d0-991c-4a20-a748-ed8c29a4142b None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:09:52.235181 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2c9fd0d0-991c-4a20-a748-ed8c29a4142b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:09:52.299971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 587/1179] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:09:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:09:52.324050 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8976e5-fc4d-4710-a487-11cbfd7726cc None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:52.376553 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:52.423729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8976e5-fc4d-4710-a487-11cbfd7726cc None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:52.457217 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934d2ae-5013-4090-8c0e-a0141dac280a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:52.457986 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934d2ae-5013-4090-8c0e-a0141dac280a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:52.458273 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934d2ae-5013-4090-8c0e-a0141dac280a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:52.458550 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934d2ae-5013-4090-8c0e-a0141dac280a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:52.468425 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8934d2ae-5013-4090-8c0e-a0141dac280a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:09:52.476006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-8934d2ae-5013-4090-8c0e-a0141dac280a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:09:52.476006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:52.476006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:52.476006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:52.476006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:52.476006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:52.476006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:52.476006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:52.476006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:52.476006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:52.476006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:52.476006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:52.476006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:09:52.476006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:09:52.476006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:09:52.476006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:09:52.477130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:09:52.477130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:09:52.477130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:09:52.477130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:09:52.477130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:52.477130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:52.477130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:52.477130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:52.477130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:52.477130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:52.477130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:52.477130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:52.477130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:52.477130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:52.477130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:52.477130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:52.478263 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:52.478263 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:52.478263 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:52.478263 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:52.478263 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:09:52.478263 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:09:52.478263 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:09:52.478263 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:09:52.478263 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:09:52.478263 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:09:52.478263 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:09:52.478263 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:09:52.478263 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 593/1180] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:09:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:09:52.483169 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9fd0d0-991c-4a20-a748-ed8c29a4142b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:52.576593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:52.579552 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9fd0d0-991c-4a20-a748-ed8c29a4142b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:52.613388 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20329a-57a2-4bd6-acce-998be4c2cf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:52.614041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20329a-57a2-4bd6-acce-998be4c2cf1e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:52.614320 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20329a-57a2-4bd6-acce-998be4c2cf1e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:52.614593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20329a-57a2-4bd6-acce-998be4c2cf1e None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:52.615359 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a20329a-57a2-4bd6-acce-998be4c2cf1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:52.615660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a20329a-57a2-4bd6-acce-998be4c2cf1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:52.617000 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a20329a-57a2-4bd6-acce-998be4c2cf1e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:52.624942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 588/1181] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:09:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:52.631194 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934d2ae-5013-4090-8c0e-a0141dac280a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:52.729096 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934d2ae-5013-4090-8c0e-a0141dac280a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:52.741807 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:52.762748 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4b943-19ab-4e85-8326-85deafd0e150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:52.763638 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4b943-19ab-4e85-8326-85deafd0e150 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:52.763924 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4b943-19ab-4e85-8326-85deafd0e150 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:52.764196 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4b943-19ab-4e85-8326-85deafd0e150 None admin] PATH_INFO: `/v3/users/1a0d6060578249a09fdc94ccb8ba7de8` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:52.776800 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4b943-19ab-4e85-8326-85deafd0e150 None admin] RBAC: Authorizing `identity:get_user(user_id=1a0d6060578249a09fdc94ccb8ba7de8)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:52.778244 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4b943-19ab-4e85-8326-85deafd0e150 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:52.779909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 594/1182] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:09:52 2022] GET /identity/v3/users/1a0d6060578249a09fdc94ccb8ba7de8 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:52.785806 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20329a-57a2-4bd6-acce-998be4c2cf1e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:52.887316 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20329a-57a2-4bd6-acce-998be4c2cf1e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:52.921632 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcfd8d2-a62c-4bac-8887-7e975b47234c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:52.922278 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcfd8d2-a62c-4bac-8887-7e975b47234c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:52.922563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcfd8d2-a62c-4bac-8887-7e975b47234c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:52.922842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcfd8d2-a62c-4bac-8887-7e975b47234c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:52.951208 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:52.988786 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcfd8d2-a62c-4bac-8887-7e975b47234c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:52.990154 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcfd8d2-a62c-4bac-8887-7e975b47234c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:53.084928 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 589/1183] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:09:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:09:53.090847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4b943-19ab-4e85-8326-85deafd0e150 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:53.189929 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4b943-19ab-4e85-8326-85deafd0e150 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:53.225662 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfedc90c-9b14-4fd6-b55f-01d860534f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:53.226525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedc90c-9b14-4fd6-b55f-01d860534f5e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:53.226809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedc90c-9b14-4fd6-b55f-01d860534f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:53.227088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedc90c-9b14-4fd6-b55f-01d860534f5e None admin] PATH_INFO: `/v3/projects/9e13bca49ebd416fa7f3ffc3eae9bcd2` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:53.228295 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfedc90c-9b14-4fd6-b55f-01d860534f5e None admin] RBAC: Authorizing `identity:get_project(project_id=9e13bca49ebd416fa7f3ffc3eae9bcd2)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:53.229721 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfedc90c-9b14-4fd6-b55f-01d860534f5e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:53.231450 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 595/1184] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:09:53 2022] GET /identity/v3/projects/9e13bca49ebd416fa7f3ffc3eae9bcd2 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:53.238249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcfd8d2-a62c-4bac-8887-7e975b47234c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:53.299081 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:53.337436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcfd8d2-a62c-4bac-8887-7e975b47234c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:53.370895 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887b567-b211-4df8-b950-eebd8ebbe75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:53.371567 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887b567-b211-4df8-b950-eebd8ebbe75f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:53.371848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887b567-b211-4df8-b950-eebd8ebbe75f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:53.372118 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887b567-b211-4df8-b950-eebd8ebbe75f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:53.372933 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887b567-b211-4df8-b950-eebd8ebbe75f None admin] RBAC: Adding query filter params (role.id=82bbe3651b9b4f378fd150d93357b2b1, scope.project.id=9e13bca49ebd416fa7f3ffc3eae9bcd2, user.id=1a0d6060578249a09fdc94ccb8ba7de8) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:53.373238 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887b567-b211-4df8-b950-eebd8ebbe75f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:53.374655 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887b567-b211-4df8-b950-eebd8ebbe75f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:53.380292 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 590/1185] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:09:53 2022] GET /identity/v3/role_assignments?user.id=1a0d6060578249a09fdc94ccb8ba7de8&scope.project.id=9e13bca49ebd416fa7f3ffc3eae9bcd2&role.id=82bbe3651b9b4f378fd150d93357b2b1 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:55.581501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dab3c-4ef1-4473-b17b-caa8cb84947b None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:55.582071 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dab3c-4ef1-4473-b17b-caa8cb84947b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:55.582071 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dab3c-4ef1-4473-b17b-caa8cb84947b None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:55.583457 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 596/1186] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:09:55.588574 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646c6a8-5def-4856-95e8-4ac9a3bb245b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:55.588995 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646c6a8-5def-4856-95e8-4ac9a3bb245b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:55.589137 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646c6a8-5def-4856-95e8-4ac9a3bb245b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:55.606505 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7646c6a8-5def-4856-95e8-4ac9a3bb245b None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:09:55.607695 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7646c6a8-5def-4856-95e8-4ac9a3bb245b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:09:55.671282 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 591/1187] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:09:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:09:55.693670 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2dab3c-4ef1-4473-b17b-caa8cb84947b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:55.735277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:55.793632 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2dab3c-4ef1-4473-b17b-caa8cb84947b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:55.825555 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7995fa1-1af1-4cae-af45-03942c6c9ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:55.826332 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7995fa1-1af1-4cae-af45-03942c6c9ffa None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:55.826614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7995fa1-1af1-4cae-af45-03942c6c9ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:55.826890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7995fa1-1af1-4cae-af45-03942c6c9ffa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:55.837278 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7995fa1-1af1-4cae-af45-03942c6c9ffa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:09:55.844777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-a7995fa1-1af1-4cae-af45-03942c6c9ffa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:09:55.844777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:55.844777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:55.844777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:55.844777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:55.844777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:55.844777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:55.844777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:55.844777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:55.844777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:55.844777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:55.844777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:55.844777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:09:55.844777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:09:55.844777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:09:55.844777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:09:55.845893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:09:55.845893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:09:55.845893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:09:55.845893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:09:55.845893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:55.845893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:55.845893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:55.845893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:55.845893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:55.845893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:55.845893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:55.845893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:55.845893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:55.845893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:55.845893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:55.845893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:55.847038 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:55.847038 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:55.847038 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:55.847038 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:55.847038 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:09:55.847038 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:09:55.847038 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:09:55.847038 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:09:55.847038 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:09:55.847038 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:09:55.847038 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:09:55.847038 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:09:55.847038 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 597/1188] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:09:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:09:55.852249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646c6a8-5def-4856-95e8-4ac9a3bb245b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:55.906579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:55.949573 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646c6a8-5def-4856-95e8-4ac9a3bb245b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:55.982749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42027f-f24a-46a4-a4d4-a7173d41ba30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:55.983503 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42027f-f24a-46a4-a4d4-a7173d41ba30 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:55.983790 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42027f-f24a-46a4-a4d4-a7173d41ba30 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:55.984095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42027f-f24a-46a4-a4d4-a7173d41ba30 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:55.984845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42027f-f24a-46a4-a4d4-a7173d41ba30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:55.985154 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42027f-f24a-46a4-a4d4-a7173d41ba30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:55.986513 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42027f-f24a-46a4-a4d4-a7173d41ba30 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:55.995255 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 592/1189] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:09:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:56.001142 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7995fa1-1af1-4cae-af45-03942c6c9ffa None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:56.094556 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:56.098984 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7995fa1-1af1-4cae-af45-03942c6c9ffa None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:56.132180 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc59fb1-5578-4717-9b4c-064cbb9e4fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:56.133036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc59fb1-5578-4717-9b4c-064cbb9e4fbd None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:56.133322 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc59fb1-5578-4717-9b4c-064cbb9e4fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:56.133590 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc59fb1-5578-4717-9b4c-064cbb9e4fbd None admin] PATH_INFO: `/v3/users/1a0d6060578249a09fdc94ccb8ba7de8` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:56.134750 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc59fb1-5578-4717-9b4c-064cbb9e4fbd None admin] RBAC: Authorizing `identity:get_user(user_id=1a0d6060578249a09fdc94ccb8ba7de8)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:56.136193 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc59fb1-5578-4717-9b4c-064cbb9e4fbd None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:56.137784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 598/1190] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:09:56 2022] GET /identity/v3/users/1a0d6060578249a09fdc94ccb8ba7de8 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:56.143468 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42027f-f24a-46a4-a4d4-a7173d41ba30 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:56.241451 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42027f-f24a-46a4-a4d4-a7173d41ba30 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:56.258196 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:56.274840 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b82086-80e7-48af-9c7b-bd8c599aa24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:56.275741 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b82086-80e7-48af-9c7b-bd8c599aa24e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:56.276029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b82086-80e7-48af-9c7b-bd8c599aa24e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:56.276308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b82086-80e7-48af-9c7b-bd8c599aa24e None admin] PATH_INFO: `/v3/projects/9e13bca49ebd416fa7f3ffc3eae9bcd2` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:56.277488 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b82086-80e7-48af-9c7b-bd8c599aa24e None admin] RBAC: Authorizing `identity:get_project(project_id=9e13bca49ebd416fa7f3ffc3eae9bcd2)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:56.278912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b82086-80e7-48af-9c7b-bd8c599aa24e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:56.280649 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 593/1191] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:09:56 2022] GET /identity/v3/projects/9e13bca49ebd416fa7f3ffc3eae9bcd2 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:56.287182 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc59fb1-5578-4717-9b4c-064cbb9e4fbd None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:56.384394 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc59fb1-5578-4717-9b4c-064cbb9e4fbd None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:56.416027 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13e359-f185-4b34-bd20-3cabd8f88af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:56.416901 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13e359-f185-4b34-bd20-3cabd8f88af0 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:56.417184 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13e359-f185-4b34-bd20-3cabd8f88af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:56.417459 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13e359-f185-4b34-bd20-3cabd8f88af0 None admin] PATH_INFO: `/v3/projects/9e13bca49ebd416fa7f3ffc3eae9bcd2/users/1a0d6060578249a09fdc94ccb8ba7de8/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:56.419512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13e359-f185-4b34-bd20-3cabd8f88af0 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e13bca49ebd416fa7f3ffc3eae9bcd2, user_id=1a0d6060578249a09fdc94ccb8ba7de8, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:56.420931 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13e359-f185-4b34-bd20-3cabd8f88af0 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:56.433571 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 599/1192] 10.222.0.81 () {58 vars in 1439 bytes} [Tue Jun 7 14:09:56 2022] PUT /identity/v3/projects/9e13bca49ebd416fa7f3ffc3eae9bcd2/users/1a0d6060578249a09fdc94ccb8ba7de8/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:09:58.637325 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5c925-1648-4dd8-91c6-3453ee271e4e None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:58.637893 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5c925-1648-4dd8-91c6-3453ee271e4e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:58.637893 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5c925-1648-4dd8-91c6-3453ee271e4e None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:58.639266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 594/1193] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:09:58.644506 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da339fac-7ec8-4cbf-9f13-c20b57c38278 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:58.644909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da339fac-7ec8-4cbf-9f13-c20b57c38278 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:58.645065 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da339fac-7ec8-4cbf-9f13-c20b57c38278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:58.661360 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-da339fac-7ec8-4cbf-9f13-c20b57c38278 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:09:58.662537 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-da339fac-7ec8-4cbf-9f13-c20b57c38278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:09:58.682701 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:58.729132 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 600/1194] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:09:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:09:58.753667 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5c925-1648-4dd8-91c6-3453ee271e4e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:58.850477 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5c925-1648-4dd8-91c6-3453ee271e4e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:58.883548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fbc58-52b9-4bbc-8b95-004cf1741d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:58.884332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fbc58-52b9-4bbc-8b95-004cf1741d6f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:58.884619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fbc58-52b9-4bbc-8b95-004cf1741d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:58.884896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fbc58-52b9-4bbc-8b95-004cf1741d6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:58.895278 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fbc58-52b9-4bbc-8b95-004cf1741d6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:09:58.903239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-9e8fbc58-52b9-4bbc-8b95-004cf1741d6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:09:58.903239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:09:58.903239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:09:58.903239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:09:58.903239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:09:58.903239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:09:58.903239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:09:58.903239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:09:58.903239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:09:58.903239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:09:58.903239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:09:58.903239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:09:58.903239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:09:58.903239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:09:58.903239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:09:58.903239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:09:58.904242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:09:58.904242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:09:58.904242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:09:58.904242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:09:58.904242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:09:58.904242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:09:58.904242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:09:58.904242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:09:58.904242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:09:58.904242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:09:58.904242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:09:58.904242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:09:58.904242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:09:58.904242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:09:58.904242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:09:58.904242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:09:58.905100 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:09:58.905100 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:09:58.905100 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:09:58.905100 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:09:58.905100 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:09:58.905100 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:09:58.905100 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:09:58.905100 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:09:58.905100 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:09:58.905100 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:09:58.905100 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:09:58.905100 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:09:58.905100 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 595/1195] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:09:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:09:58.910593 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da339fac-7ec8-4cbf-9f13-c20b57c38278 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:59.006651 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da339fac-7ec8-4cbf-9f13-c20b57c38278 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:59.039734 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629c2fb-bc66-45c5-b207-75832ae75da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:59.040388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629c2fb-bc66-45c5-b207-75832ae75da6 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:59.040668 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629c2fb-bc66-45c5-b207-75832ae75da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:59.040941 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629c2fb-bc66-45c5-b207-75832ae75da6 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:59.041685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629c2fb-bc66-45c5-b207-75832ae75da6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:59.041986 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629c2fb-bc66-45c5-b207-75832ae75da6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:59.043352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629c2fb-bc66-45c5-b207-75832ae75da6 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:59.051183 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 601/1196] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:09:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:59.056812 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fbc58-52b9-4bbc-8b95-004cf1741d6f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:59.102407 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:59.154682 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fbc58-52b9-4bbc-8b95-004cf1741d6f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:59.188485 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac1dd4-a3ec-41f9-b3c8-d97fe636154f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:59.189333 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac1dd4-a3ec-41f9-b3c8-d97fe636154f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:59.190225 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac1dd4-a3ec-41f9-b3c8-d97fe636154f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:59.190718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac1dd4-a3ec-41f9-b3c8-d97fe636154f None admin] PATH_INFO: `/v3/users/1a0d6060578249a09fdc94ccb8ba7de8` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:59.191888 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac1dd4-a3ec-41f9-b3c8-d97fe636154f None admin] RBAC: Authorizing `identity:get_user(user_id=1a0d6060578249a09fdc94ccb8ba7de8)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:59.193316 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac1dd4-a3ec-41f9-b3c8-d97fe636154f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:59.194907 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 596/1197] 10.222.0.81 () {56 vars in 1177 bytes} [Tue Jun 7 14:09:59 2022] GET /identity/v3/users/1a0d6060578249a09fdc94ccb8ba7de8 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:59.200611 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629c2fb-bc66-45c5-b207-75832ae75da6 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:59.219882 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:59.298725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629c2fb-bc66-45c5-b207-75832ae75da6 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:59.332979 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033c7fd-54a1-4b1e-80b9-ec5cfb7d33ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:59.333617 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033c7fd-54a1-4b1e-80b9-ec5cfb7d33ec None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:59.333898 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033c7fd-54a1-4b1e-80b9-ec5cfb7d33ec None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:59.334172 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033c7fd-54a1-4b1e-80b9-ec5cfb7d33ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:59.399186 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033c7fd-54a1-4b1e-80b9-ec5cfb7d33ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:59.400653 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033c7fd-54a1-4b1e-80b9-ec5cfb7d33ec None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:59.417984 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:59.495596 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 602/1198] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:09:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:09:59.501487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac1dd4-a3ec-41f9-b3c8-d97fe636154f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:59.597066 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac1dd4-a3ec-41f9-b3c8-d97fe636154f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:59.610269 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:59.630737 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9913b009-6fb2-4c52-815c-26041c6d65ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:59.631606 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913b009-6fb2-4c52-815c-26041c6d65ee None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:59.631890 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913b009-6fb2-4c52-815c-26041c6d65ee None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:59.632164 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913b009-6fb2-4c52-815c-26041c6d65ee None admin] PATH_INFO: `/v3/projects/9e13bca49ebd416fa7f3ffc3eae9bcd2` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:59.633321 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9913b009-6fb2-4c52-815c-26041c6d65ee None admin] RBAC: Authorizing `identity:get_project(project_id=9e13bca49ebd416fa7f3ffc3eae9bcd2)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:59.634732 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9913b009-6fb2-4c52-815c-26041c6d65ee None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:59.636451 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 597/1199] 10.222.0.81 () {56 vars in 1186 bytes} [Tue Jun 7 14:09:59 2022] GET /identity/v3/projects/9e13bca49ebd416fa7f3ffc3eae9bcd2 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:09:59.642585 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033c7fd-54a1-4b1e-80b9-ec5cfb7d33ec None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:09:59.737650 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033c7fd-54a1-4b1e-80b9-ec5cfb7d33ec None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:09:59.755120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:09:59.772513 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319b24f-36fa-4c2f-ae87-a3634fa62825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:09:59.773182 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319b24f-36fa-4c2f-ae87-a3634fa62825 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:09:59.773465 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319b24f-36fa-4c2f-ae87-a3634fa62825 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:09:59.773739 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319b24f-36fa-4c2f-ae87-a3634fa62825 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:09:59.774562 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7319b24f-36fa-4c2f-ae87-a3634fa62825 None admin] RBAC: Adding query filter params (role.id=82bbe3651b9b4f378fd150d93357b2b1, scope.project.id=9e13bca49ebd416fa7f3ffc3eae9bcd2, user.id=1a0d6060578249a09fdc94ccb8ba7de8) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:09:59.774870 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7319b24f-36fa-4c2f-ae87-a3634fa62825 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:09:59.776347 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7319b24f-36fa-4c2f-ae87-a3634fa62825 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:09:59.782005 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 603/1200] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:09:59 2022] GET /identity/v3/role_assignments?user.id=1a0d6060578249a09fdc94ccb8ba7de8&scope.project.id=9e13bca49ebd416fa7f3ffc3eae9bcd2&role.id=82bbe3651b9b4f378fd150d93357b2b1 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:10:09.626241 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=9 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:10:09.779427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=9 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:01.038272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb6b12-f92d-4c49-9167-bd8551ff264f None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:01.038272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb6b12-f92d-4c49-9167-bd8551ff264f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:01.038272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb6b12-f92d-4c49-9167-bd8551ff264f None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:01.038272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 598/1201] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:11:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:11:01.044077 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab994f7-5b97-4e52-80dd-db4507a964a4 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:01.044533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab994f7-5b97-4e52-80dd-db4507a964a4 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:01.044628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab994f7-5b97-4e52-80dd-db4507a964a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:01.062588 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-5ab994f7-5b97-4e52-80dd-db4507a964a4 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:11:01.064214 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5ab994f7-5b97-4e52-80dd-db4507a964a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:11:01.151713 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 604/1202] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:11:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:11:01.179126 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb6b12-f92d-4c49-9167-bd8551ff264f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:01.304854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb6b12-f92d-4c49-9167-bd8551ff264f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:01.351344 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78508f9e-5172-4c4f-a90b-32a93c41f446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:01.352180 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78508f9e-5172-4c4f-a90b-32a93c41f446 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:01.352465 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78508f9e-5172-4c4f-a90b-32a93c41f446 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:01.352754 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78508f9e-5172-4c4f-a90b-32a93c41f446 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:01.364959 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78508f9e-5172-4c4f-a90b-32a93c41f446 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:01.366677 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78508f9e-5172-4c4f-a90b-32a93c41f446 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:01.386680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-78508f9e-5172-4c4f-a90b-32a93c41f446 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:11:01.386680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:01.386680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:01.386680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:01.386680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:01.386680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:01.386680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:01.386680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:01.386680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:01.386680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:01.386680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:01.386680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:01.386680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:11:01.386680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:11:01.386680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:11:01.387795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:11:01.387795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:01.387795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:01.387795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:01.387795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:01.387795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:01.387795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:01.387795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:01.387795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:01.387795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:01.387795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:01.387795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:01.387795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:01.387795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:01.387795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:01.387795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:01.387795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:01.388864 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:11:01.388864 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:11:01.388864 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:11:01.388864 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:11:01.395255 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 599/1203] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:11:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:01.395443 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab994f7-5b97-4e52-80dd-db4507a964a4 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:01.514722 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab994f7-5b97-4e52-80dd-db4507a964a4 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:01.554037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:01.559665 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0df717-bec5-4f7e-8cf0-c78e05c728f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:01.560975 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0df717-bec5-4f7e-8cf0-c78e05c728f0 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:01.561625 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0df717-bec5-4f7e-8cf0-c78e05c728f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:01.562804 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0df717-bec5-4f7e-8cf0-c78e05c728f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:01.564322 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0df717-bec5-4f7e-8cf0-c78e05c728f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:01.565024 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0df717-bec5-4f7e-8cf0-c78e05c728f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:01.566459 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0df717-bec5-4f7e-8cf0-c78e05c728f0 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:01.583434 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 605/1204] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:11:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:01.585462 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78508f9e-5172-4c4f-a90b-32a93c41f446 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:01.661087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:01.726077 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78508f9e-5172-4c4f-a90b-32a93c41f446 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:01.779423 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f2ab7-affb-47e8-b31f-a9eef05ba361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:01.780422 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f2ab7-affb-47e8-b31f-a9eef05ba361 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:01.781541 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f2ab7-affb-47e8-b31f-a9eef05ba361 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:01.781890 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f2ab7-affb-47e8-b31f-a9eef05ba361 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:01.782648 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f2ab7-affb-47e8-b31f-a9eef05ba361 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:01.784222 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f2ab7-affb-47e8-b31f-a9eef05ba361 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:01.810905 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 600/1205] 10.222.0.81 () {60 vars in 1132 bytes} [Tue Jun 7 14:11:01 2022] POST /identity/v3/users => generated 251 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:11:04.471527 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb3b40-38e1-43e3-8f26-da616a01ccf3 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:04.472409 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb3b40-38e1-43e3-8f26-da616a01ccf3 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:04.472409 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb3b40-38e1-43e3-8f26-da616a01ccf3 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:04.473468 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 606/1206] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:11:04.478774 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b760d-425f-4e01-9db4-4750765068b6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:04.479241 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b760d-425f-4e01-9db4-4750765068b6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:04.479439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b760d-425f-4e01-9db4-4750765068b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:04.498808 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-114b760d-425f-4e01-9db4-4750765068b6 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:11:04.499261 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-114b760d-425f-4e01-9db4-4750765068b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:11:04.591953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 601/1207] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:11:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:11:04.616793 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb3b40-38e1-43e3-8f26-da616a01ccf3 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:04.750945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb3b40-38e1-43e3-8f26-da616a01ccf3 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:04.794899 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57118aa-dd6d-4dfc-b363-4b3e4b698998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:04.795767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57118aa-dd6d-4dfc-b363-4b3e4b698998 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:04.796079 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57118aa-dd6d-4dfc-b363-4b3e4b698998 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:04.796813 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57118aa-dd6d-4dfc-b363-4b3e4b698998 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:04.808936 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57118aa-dd6d-4dfc-b363-4b3e4b698998 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:11:04.818230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-b57118aa-dd6d-4dfc-b363-4b3e4b698998 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:11:04.818230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:04.818230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:04.818230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:04.818230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:04.818230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:04.818230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:04.818230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:04.818230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:04.818230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:04.818230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:04.818230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:04.818230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:11:04.818230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:11:04.818230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:11:04.818230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:11:04.819820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:11:04.819820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:11:04.819820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:11:04.819820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:11:04.819820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:04.819820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:04.819820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:04.819820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:04.819820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:04.819820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:04.819820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:04.819820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:04.819820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:04.819820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:04.819820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:04.819820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:04.821305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:04.821305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:04.821305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:04.821305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:04.821305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:11:04.821305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:11:04.821305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:11:04.821305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:11:04.821305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:11:04.821305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:11:04.821305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:11:04.821305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:11:04.821305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 607/1208] 10.222.0.81 () {56 vars in 1102 bytes} [Tue Jun 7 14:11:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:04.825745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b760d-425f-4e01-9db4-4750765068b6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:04.876256 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:04.964196 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b760d-425f-4e01-9db4-4750765068b6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:05.012378 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146b776-3a3b-4d7a-a6ab-1697510026ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:05.012378 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146b776-3a3b-4d7a-a6ab-1697510026ef None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:05.012378 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146b776-3a3b-4d7a-a6ab-1697510026ef None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:05.012378 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146b776-3a3b-4d7a-a6ab-1697510026ef None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:05.012817 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146b776-3a3b-4d7a-a6ab-1697510026ef None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:05.013150 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146b776-3a3b-4d7a-a6ab-1697510026ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:05.014490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146b776-3a3b-4d7a-a6ab-1697510026ef None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:05.023970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 602/1209] 10.222.0.81 () {56 vars in 1103 bytes} [Tue Jun 7 14:11:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:11:05.029809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57118aa-dd6d-4dfc-b363-4b3e4b698998 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:05.090622 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:05.151436 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57118aa-dd6d-4dfc-b363-4b3e4b698998 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:05.194131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43451300-c256-4999-9a9e-4190a5546843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:05.195203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43451300-c256-4999-9a9e-4190a5546843 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:05.195508 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43451300-c256-4999-9a9e-4190a5546843 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:05.195882 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43451300-c256-4999-9a9e-4190a5546843 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:05.208479 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43451300-c256-4999-9a9e-4190a5546843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:05.210310 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43451300-c256-4999-9a9e-4190a5546843 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:05.230725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-43451300-c256-4999-9a9e-4190a5546843 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:11:05.230725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:05.230725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:05.230725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:05.230725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:05.230725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:05.230725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:05.230725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:05.230725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:05.230725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:05.230725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:05.230725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:05.230725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:11:05.230725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:11:05.230725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:11:05.231966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:11:05.231966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:05.231966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:05.231966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:05.231966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:05.231966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:05.231966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:05.231966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:05.231966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:05.231966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:05.231966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:05.231966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:05.231966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:05.231966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:05.231966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:05.231966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:05.231966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:05.238284 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:11:05.238284 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:11:05.238284 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:11:05.238284 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:11:05.238284 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 608/1210] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:11:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:11:05.238579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146b776-3a3b-4d7a-a6ab-1697510026ef None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:05.357306 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146b776-3a3b-4d7a-a6ab-1697510026ef None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:05.359354 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:05.400119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72274901-2c19-4463-908d-bd85d6e9392f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:05.401111 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72274901-2c19-4463-908d-bd85d6e9392f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:05.401505 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72274901-2c19-4463-908d-bd85d6e9392f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:05.401927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72274901-2c19-4463-908d-bd85d6e9392f None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:05.402839 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72274901-2c19-4463-908d-bd85d6e9392f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:05.403280 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72274901-2c19-4463-908d-bd85d6e9392f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:05.405005 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72274901-2c19-4463-908d-bd85d6e9392f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:05.418734 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 603/1211] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:05.425283 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43451300-c256-4999-9a9e-4190a5546843 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:05.551458 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43451300-c256-4999-9a9e-4190a5546843 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:05.553242 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:05.592665 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a0363-fd49-43b8-9695-e9ac7babcf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:05.593619 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a0363-fd49-43b8-9695-e9ac7babcf82 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:05.593900 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a0363-fd49-43b8-9695-e9ac7babcf82 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:05.594173 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a0363-fd49-43b8-9695-e9ac7babcf82 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:05.602883 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a0363-fd49-43b8-9695-e9ac7babcf82 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:05.604616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a0363-fd49-43b8-9695-e9ac7babcf82 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:05.621918 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-cc8a0363-fd49-43b8-9695-e9ac7babcf82 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 14:11:05.621918 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:05.621918 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:05.621918 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:05.621918 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:05.621918 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:05.621918 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:05.621918 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:05.621918 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:05.621918 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:05.621918 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:05.621918 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:05.621918 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:11:05.621918 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:11:05.621918 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:11:05.621918 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:11:05.622988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:05.622988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:05.622988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:11:05.622988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:11:05.622988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:11:05.622988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:11:05.622988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:05.622988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:05.622988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:05.622988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:05.622988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:05.622988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:05.622988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:05.622988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:05.622988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:05.622988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:05.622988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:05.622988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:05.624002 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:05.624002 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:05.624002 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:11:05.624002 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:11:05.624002 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:11:05.624002 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:11:05.624002 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:11:05.624002 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:11:05.624002 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 14:11:05.624002 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:11:05.624002 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 609/1212] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:11:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:11:05.628979 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72274901-2c19-4463-908d-bd85d6e9392f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:05.751534 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72274901-2c19-4463-908d-bd85d6e9392f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:05.792387 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece08867-9311-4b3a-9e1c-49848ed6c936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:05.793137 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece08867-9311-4b3a-9e1c-49848ed6c936 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:05.793460 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece08867-9311-4b3a-9e1c-49848ed6c936 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:05.793801 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece08867-9311-4b3a-9e1c-49848ed6c936 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:05.794742 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece08867-9311-4b3a-9e1c-49848ed6c936 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:05.795087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece08867-9311-4b3a-9e1c-49848ed6c936 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:05.796791 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece08867-9311-4b3a-9e1c-49848ed6c936 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:05.820869 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 604/1213] 10.222.0.81 () {56 vars in 1143 bytes} [Tue Jun 7 14:11:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:11:05.828741 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a0363-fd49-43b8-9695-e9ac7babcf82 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:05.958556 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a0363-fd49-43b8-9695-e9ac7babcf82 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:06.002637 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a10cba-5781-499f-ac98-587de29fcc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:06.003347 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a10cba-5781-499f-ac98-587de29fcc72 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:06.003621 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a10cba-5781-499f-ac98-587de29fcc72 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:06.003910 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a10cba-5781-499f-ac98-587de29fcc72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:06.043649 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:06.100231 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a10cba-5781-499f-ac98-587de29fcc72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:06.101691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a10cba-5781-499f-ac98-587de29fcc72 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:06.225095 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 610/1214] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:11:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:11:06.230930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece08867-9311-4b3a-9e1c-49848ed6c936 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:06.253449 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:06.349691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece08867-9311-4b3a-9e1c-49848ed6c936 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:06.389015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8262f-3e0c-4e30-87ed-7319c4f7ace6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:06.390648 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8262f-3e0c-4e30-87ed-7319c4f7ace6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:06.391067 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8262f-3e0c-4e30-87ed-7319c4f7ace6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:06.391644 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8262f-3e0c-4e30-87ed-7319c4f7ace6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:06.404575 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f8262f-3e0c-4e30-87ed-7319c4f7ace6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:06.407002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f8262f-3e0c-4e30-87ed-7319c4f7ace6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:06.428346 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-b2f8262f-3e0c-4e30-87ed-7319c4f7ace6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:11:06.428346 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:06.428346 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:06.428346 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:06.428346 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:06.428346 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:06.428346 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:06.428346 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:06.428346 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:06.428346 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:06.428346 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:06.428346 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:06.428346 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:11:06.428346 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:11:06.428346 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:11:06.429427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:11:06.429427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:06.429427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:06.429427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:06.429427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:06.429427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:06.429427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:06.429427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:06.429427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:06.429427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:06.429427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:06.429427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:06.429427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:06.429427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:06.429427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:06.429427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:06.429427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:06.430397 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:11:06.430397 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:11:06.430397 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:11:06.430397 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:11:06.430397 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 605/1215] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:11:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:11:06.435232 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a10cba-5781-499f-ac98-587de29fcc72 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:06.503015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:06.548189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a10cba-5781-499f-ac98-587de29fcc72 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:06.586015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94003f-7893-484c-933e-21146289a09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:06.586808 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94003f-7893-484c-933e-21146289a09a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:06.587150 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94003f-7893-484c-933e-21146289a09a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:06.587502 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94003f-7893-484c-933e-21146289a09a None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:06.588295 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc94003f-7893-484c-933e-21146289a09a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:06.588708 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc94003f-7893-484c-933e-21146289a09a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:06.590110 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc94003f-7893-484c-933e-21146289a09a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:06.602636 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 611/1216] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:11:06.612500 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8262f-3e0c-4e30-87ed-7319c4f7ace6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:06.692933 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:06.726285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8262f-3e0c-4e30-87ed-7319c4f7ace6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:06.761788 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd8641-2fec-413f-a2c5-d8efebea159e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:06.762596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd8641-2fec-413f-a2c5-d8efebea159e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:06.762942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd8641-2fec-413f-a2c5-d8efebea159e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:06.763311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd8641-2fec-413f-a2c5-d8efebea159e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:06.770804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd8641-2fec-413f-a2c5-d8efebea159e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:06.772392 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd8641-2fec-413f-a2c5-d8efebea159e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:06.786386 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f2bd8641-2fec-413f-a2c5-d8efebea159e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:11:06.786386 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:06.786386 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:06.786386 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:06.786386 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:06.786386 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:06.786386 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:06.786386 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:06.786386 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:06.786386 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:06.786386 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:06.786386 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:06.786386 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:11:06.786386 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:11:06.786386 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:11:06.787680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:11:06.787680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:06.787680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:06.787680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:06.787680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:06.787680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:06.787680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:06.787680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:06.787680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:06.787680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:06.787680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:06.787680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:06.787680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:06.787680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:06.787680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:06.787680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:06.787680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:06.788821 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:11:06.788821 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:11:06.788821 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:11:06.788821 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:11:06.788821 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:11:06.788821 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:11:06.788821 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:11:06.788821 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:11:06.789379 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 606/1217] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:11:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:06.795521 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94003f-7893-484c-933e-21146289a09a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:06.918070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94003f-7893-484c-933e-21146289a09a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:06.949571 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:06.964674 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a6a9db-e99c-4344-98af-35c147f81a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:06.965436 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6a9db-e99c-4344-98af-35c147f81a13 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:06.965762 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6a9db-e99c-4344-98af-35c147f81a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:06.966071 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6a9db-e99c-4344-98af-35c147f81a13 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:06.966990 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a6a9db-e99c-4344-98af-35c147f81a13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:06.967398 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a6a9db-e99c-4344-98af-35c147f81a13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:06.969017 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a6a9db-e99c-4344-98af-35c147f81a13 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:06.987477 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 612/1218] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:11:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:06.994210 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd8641-2fec-413f-a2c5-d8efebea159e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:07.111983 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd8641-2fec-413f-a2c5-d8efebea159e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:07.145795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:07.153212 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0da52-f475-4e5a-8329-abc603ca32c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:07.153212 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0da52-f475-4e5a-8329-abc603ca32c1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:07.153489 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0da52-f475-4e5a-8329-abc603ca32c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:07.153743 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0da52-f475-4e5a-8329-abc603ca32c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:07.154661 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b0da52-f475-4e5a-8329-abc603ca32c1 None admin] RBAC: Adding query filter params (role.id=a72e5464b7b746978805326c224753bc, scope.project.id=f67bad575f8b4a47bee7c314497cb2fe, user.id=ab73cf3febbf4c768cd0116306f16c50) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:07.155034 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b0da52-f475-4e5a-8329-abc603ca32c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:07.156791 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b0da52-f475-4e5a-8329-abc603ca32c1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:07.164007 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 607/1219] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:11:06 2022] GET /identity/v3/role_assignments?user.id=ab73cf3febbf4c768cd0116306f16c50&scope.project.id=f67bad575f8b4a47bee7c314497cb2fe&role.id=a72e5464b7b746978805326c224753bc => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:09.657694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9c18c-aaf9-422a-92bf-23ccda6393e0 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:09.658270 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9c18c-aaf9-422a-92bf-23ccda6393e0 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:09.658270 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9c18c-aaf9-422a-92bf-23ccda6393e0 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:09.659930 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 613/1220] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:11:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:11:09.666591 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb9bee-120d-47f9-93e9-cee2a507b92c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:09.667070 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb9bee-120d-47f9-93e9-cee2a507b92c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:09.667160 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb9bee-120d-47f9-93e9-cee2a507b92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:09.685651 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-56bb9bee-120d-47f9-93e9-cee2a507b92c None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:11:09.687997 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-56bb9bee-120d-47f9-93e9-cee2a507b92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:11:09.764594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 608/1221] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:11:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:11:09.787819 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b9c18c-aaf9-422a-92bf-23ccda6393e0 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:09.903058 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b9c18c-aaf9-422a-92bf-23ccda6393e0 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:09.943484 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91de5a9-e513-41ea-aa41-557e1315d241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:09.944208 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91de5a9-e513-41ea-aa41-557e1315d241 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:09.944494 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91de5a9-e513-41ea-aa41-557e1315d241 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:09.944774 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91de5a9-e513-41ea-aa41-557e1315d241 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:09.955822 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91de5a9-e513-41ea-aa41-557e1315d241 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:11:09.963852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-f91de5a9-e513-41ea-aa41-557e1315d241 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:11:09.963852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:09.963852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:09.963852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:09.963852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:09.963852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:09.963852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:09.963852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:09.963852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:09.963852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:09.963852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:09.963852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:09.963852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:11:09.963852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:11:09.963852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:11:09.963852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:11:09.965048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:11:09.965048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:11:09.965048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:11:09.965048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:11:09.965048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:09.965048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:09.965048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:09.965048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:09.965048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:09.965048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:09.965048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:09.965048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:09.965048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:09.965048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:09.965048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:09.965048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:09.966127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:09.966127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:09.966127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:09.966127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:09.966127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:11:09.966127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:11:09.966127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:11:09.966127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:11:09.966127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:11:09.966127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:11:09.966127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:11:09.966127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:11:09.969990 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 614/1222] 10.222.0.81 () {56 vars in 1102 bytes} [Tue Jun 7 14:11:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:09.971245 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb9bee-120d-47f9-93e9-cee2a507b92c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:10.082966 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb9bee-120d-47f9-93e9-cee2a507b92c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:10.112660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:10.126557 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34a5c4-a02d-4c74-a7b4-2c13ddb16fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:10.127258 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34a5c4-a02d-4c74-a7b4-2c13ddb16fcf None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:10.127545 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34a5c4-a02d-4c74-a7b4-2c13ddb16fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:10.127816 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34a5c4-a02d-4c74-a7b4-2c13ddb16fcf None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:10.128565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34a5c4-a02d-4c74-a7b4-2c13ddb16fcf None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:10.129016 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34a5c4-a02d-4c74-a7b4-2c13ddb16fcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:10.130387 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34a5c4-a02d-4c74-a7b4-2c13ddb16fcf None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:10.138890 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 609/1223] 10.222.0.81 () {56 vars in 1103 bytes} [Tue Jun 7 14:11:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:10.145229 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91de5a9-e513-41ea-aa41-557e1315d241 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:10.194988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:10.265424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91de5a9-e513-41ea-aa41-557e1315d241 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:10.445152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7513ba-207b-489b-977a-211a43659d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:10.445919 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7513ba-207b-489b-977a-211a43659d8f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:10.446209 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7513ba-207b-489b-977a-211a43659d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:10.446611 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7513ba-207b-489b-977a-211a43659d8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:10.457332 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7513ba-207b-489b-977a-211a43659d8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:10.458772 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7513ba-207b-489b-977a-211a43659d8f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:10.479717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-5f7513ba-207b-489b-977a-211a43659d8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:11:10.479717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:10.479717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:10.479717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:10.479717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:10.479717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:10.479717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:10.479717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:10.479717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:10.479717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:10.479717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:10.479717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:10.479717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:11:10.479717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:11:10.479717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:11:10.481208 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:11:10.481208 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:10.481208 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:10.481208 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:10.481208 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:10.481208 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:10.481208 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:10.481208 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:10.481208 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:10.481208 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:10.481208 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:10.481208 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:10.481208 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:10.481208 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:10.481208 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:10.481208 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:10.481208 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:10.482694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:11:10.482694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:11:10.482694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:11:10.482694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:11:10.482694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 615/1224] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:11:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:10.488497 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34a5c4-a02d-4c74-a7b4-2c13ddb16fcf None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:10.598984 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34a5c4-a02d-4c74-a7b4-2c13ddb16fcf None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:10.634610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d808c32-8d74-44a6-bd6e-42d809ca5530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:10.635316 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d808c32-8d74-44a6-bd6e-42d809ca5530 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:10.635606 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d808c32-8d74-44a6-bd6e-42d809ca5530 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:10.635887 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d808c32-8d74-44a6-bd6e-42d809ca5530 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:10.636621 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d808c32-8d74-44a6-bd6e-42d809ca5530 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:10.636930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d808c32-8d74-44a6-bd6e-42d809ca5530 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:10.638284 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d808c32-8d74-44a6-bd6e-42d809ca5530 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:10.651763 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 610/1225] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:10.657273 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7513ba-207b-489b-977a-211a43659d8f None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:10.764729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:10.772500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7513ba-207b-489b-977a-211a43659d8f None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:10.810591 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab9908-eee5-40a4-911d-ba0145ae9e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:10.811426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab9908-eee5-40a4-911d-ba0145ae9e77 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:10.811774 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab9908-eee5-40a4-911d-ba0145ae9e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:10.812641 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab9908-eee5-40a4-911d-ba0145ae9e77 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:10.819980 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab9908-eee5-40a4-911d-ba0145ae9e77 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:10.821464 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab9908-eee5-40a4-911d-ba0145ae9e77 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:10.836149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-83ab9908-eee5-40a4-911d-ba0145ae9e77 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 14:11:10.836149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:10.836149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:10.836149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:10.836149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:10.836149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:10.836149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:10.836149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:10.836149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:10.836149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:10.836149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:10.836149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:10.836149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:11:10.836149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:11:10.836149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:11:10.836149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:11:10.837490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:10.837490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:10.837490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:11:10.837490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:11:10.837490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:11:10.837490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:11:10.837490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:10.837490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:10.837490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:10.837490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:10.837490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:10.837490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:10.837490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:10.837490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:10.837490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:10.837490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:10.837490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:10.837490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:10.838676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:10.838676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:10.838676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:11:10.838676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:11:10.838676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:11:10.838676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:11:10.838676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:11:10.838676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:11:10.838676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 14:11:10.838676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:11:10.838676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 616/1226] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:11:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:10.843871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d808c32-8d74-44a6-bd6e-42d809ca5530 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:10.896745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:10.958107 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d808c32-8d74-44a6-bd6e-42d809ca5530 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:11.002284 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149873d-e9ec-4fce-bce4-35d9c61e4e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:11.003114 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149873d-e9ec-4fce-bce4-35d9c61e4e33 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:11.003678 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149873d-e9ec-4fce-bce4-35d9c61e4e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:11.004104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149873d-e9ec-4fce-bce4-35d9c61e4e33 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:11.005105 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149873d-e9ec-4fce-bce4-35d9c61e4e33 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:11.005495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149873d-e9ec-4fce-bce4-35d9c61e4e33 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:11.007099 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149873d-e9ec-4fce-bce4-35d9c61e4e33 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:11.025902 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 611/1227] 10.222.0.81 () {56 vars in 1143 bytes} [Tue Jun 7 14:11:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:11.032474 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab9908-eee5-40a4-911d-ba0145ae9e77 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:11.146242 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab9908-eee5-40a4-911d-ba0145ae9e77 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:11.184855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fa2f6-42dc-4d19-88c4-0e19ee3008e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:11.185622 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fa2f6-42dc-4d19-88c4-0e19ee3008e0 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:11.185908 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fa2f6-42dc-4d19-88c4-0e19ee3008e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:11.186167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fa2f6-42dc-4d19-88c4-0e19ee3008e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:11.201981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8fa2f6-42dc-4d19-88c4-0e19ee3008e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:11.204064 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8fa2f6-42dc-4d19-88c4-0e19ee3008e0 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:11.221747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-fa8fa2f6-42dc-4d19-88c4-0e19ee3008e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:11:11.221747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:11.221747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:11.221747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:11.221747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:11.221747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:11.221747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:11.221747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:11.221747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:11.221747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:11.221747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:11.221747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:11.221747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:11:11.221747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:11:11.221747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:11:11.222724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:11:11.222724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:11.222724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:11.222724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:11.222724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:11.222724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:11.222724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:11.222724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:11.222724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:11.222724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:11.222724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:11.222724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:11.222724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:11.222724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:11.222724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:11.222724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:11.222724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:11.223811 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:11:11.223811 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:11:11.223811 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:11:11.223811 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:11:11.223811 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 617/1228] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:11:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:11:11.228646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149873d-e9ec-4fce-bce4-35d9c61e4e33 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:11.337699 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149873d-e9ec-4fce-bce4-35d9c61e4e33 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:11.340167 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:11.378684 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc714d-bff9-452d-ab8b-25c88f9c2a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:11.379382 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc714d-bff9-452d-ab8b-25c88f9c2a55 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:11.379679 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc714d-bff9-452d-ab8b-25c88f9c2a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:11.379968 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc714d-bff9-452d-ab8b-25c88f9c2a55 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:11.380665 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc714d-bff9-452d-ab8b-25c88f9c2a55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:11.380977 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc714d-bff9-452d-ab8b-25c88f9c2a55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:11.382385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc714d-bff9-452d-ab8b-25c88f9c2a55 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:11.395798 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 612/1229] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:11.401338 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fa2f6-42dc-4d19-88c4-0e19ee3008e0 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:11.410431 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:11.507665 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fa2f6-42dc-4d19-88c4-0e19ee3008e0 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:11.545801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5062d1-9677-45df-bfe1-ecf311e3762e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:11.547260 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5062d1-9677-45df-bfe1-ecf311e3762e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:11.547571 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5062d1-9677-45df-bfe1-ecf311e3762e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:11.547847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5062d1-9677-45df-bfe1-ecf311e3762e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:11.555764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5062d1-9677-45df-bfe1-ecf311e3762e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:11.557258 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5062d1-9677-45df-bfe1-ecf311e3762e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:11.573645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-aa5062d1-9677-45df-bfe1-ecf311e3762e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:11:11.573645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:11.573645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:11.573645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:11.573645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:11.573645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:11.573645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:11.573645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:11.573645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:11.573645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:11.573645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:11.573645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:11.573645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:11:11.573645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:11:11.573645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:11:11.574642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:11:11.574642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:11.574642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:11.574642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:11.574642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:11.574642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:11.574642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:11.574642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:11.574642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:11.574642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:11.574642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:11.574642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:11.574642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:11.574642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:11.574642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:11.574642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:11.574642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:11.575613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:11:11.575613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:11:11.575613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:11:11.575613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:11:11.575613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:11:11.575613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:11:11.575613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:11:11.575613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:11:11.575613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 618/1230] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:11:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:11.581690 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc714d-bff9-452d-ab8b-25c88f9c2a55 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:11.694042 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc714d-bff9-452d-ab8b-25c88f9c2a55 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:11.729765 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc024eb-fe58-4c3f-a9b9-438dc1a9773e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:11.730433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc024eb-fe58-4c3f-a9b9-438dc1a9773e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:11.730709 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc024eb-fe58-4c3f-a9b9-438dc1a9773e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:11.731026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc024eb-fe58-4c3f-a9b9-438dc1a9773e None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:11.731932 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc024eb-fe58-4c3f-a9b9-438dc1a9773e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:11.732295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc024eb-fe58-4c3f-a9b9-438dc1a9773e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:11.734504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc024eb-fe58-4c3f-a9b9-438dc1a9773e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:11.747857 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 613/1231] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:11:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:11.754655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5062d1-9677-45df-bfe1-ecf311e3762e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:11.827085 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:11.865046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5062d1-9677-45df-bfe1-ecf311e3762e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:11.901214 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40961320-9c0e-4a1b-84ab-ae7ff68684f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:11.902239 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40961320-9c0e-4a1b-84ab-ae7ff68684f4 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:11.902596 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40961320-9c0e-4a1b-84ab-ae7ff68684f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:11.902966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40961320-9c0e-4a1b-84ab-ae7ff68684f4 None admin] PATH_INFO: `/v3/projects/f67bad575f8b4a47bee7c314497cb2fe/users/ab73cf3febbf4c768cd0116306f16c50/roles/a72e5464b7b746978805326c224753bc` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:11.923082 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40961320-9c0e-4a1b-84ab-ae7ff68684f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f67bad575f8b4a47bee7c314497cb2fe, user_id=ab73cf3febbf4c768cd0116306f16c50, role_id=a72e5464b7b746978805326c224753bc)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:11.924599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40961320-9c0e-4a1b-84ab-ae7ff68684f4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:11.939816 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 619/1232] 10.222.0.81 () {58 vars in 1439 bytes} [Tue Jun 7 14:11:11 2022] PUT /identity/v3/projects/f67bad575f8b4a47bee7c314497cb2fe/users/ab73cf3febbf4c768cd0116306f16c50/roles/a72e5464b7b746978805326c224753bc => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:11:14.388944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74def6bc-465b-47b8-9864-6d03f7d3a5ec None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:14.389492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74def6bc-465b-47b8-9864-6d03f7d3a5ec None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:14.389492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74def6bc-465b-47b8-9864-6d03f7d3a5ec None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:14.391629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 614/1233] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:11:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:11:14.397152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869668d8-dd70-46e6-9095-8d2c59ffa8a9 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:14.397615 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869668d8-dd70-46e6-9095-8d2c59ffa8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:14.397707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869668d8-dd70-46e6-9095-8d2c59ffa8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:14.416176 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-869668d8-dd70-46e6-9095-8d2c59ffa8a9 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:11:14.417514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-869668d8-dd70-46e6-9095-8d2c59ffa8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:11:14.491035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 620/1234] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:11:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:11:14.515416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74def6bc-465b-47b8-9864-6d03f7d3a5ec None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:14.541121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:14.628027 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74def6bc-465b-47b8-9864-6d03f7d3a5ec None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:14.665829 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d079d15-8b24-4642-89b3-c074dbc4479b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:14.666680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d079d15-8b24-4642-89b3-c074dbc4479b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:14.667041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d079d15-8b24-4642-89b3-c074dbc4479b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:14.667395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d079d15-8b24-4642-89b3-c074dbc4479b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:14.680171 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d079d15-8b24-4642-89b3-c074dbc4479b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:11:14.690131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-7d079d15-8b24-4642-89b3-c074dbc4479b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:11:14.690131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:14.690131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:14.690131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:14.690131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:14.690131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:14.690131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:14.690131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:14.690131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:14.690131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:14.690131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:14.690131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:14.690131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:11:14.690131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:11:14.690131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:11:14.690131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:11:14.691395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:11:14.691395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:11:14.691395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:11:14.691395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:11:14.691395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:14.691395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:14.691395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:14.691395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:14.691395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:14.691395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:14.691395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:14.691395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:14.691395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:14.691395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:14.691395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:14.691395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:14.692275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:14.692275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:14.692275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:14.692275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:14.692275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:11:14.692275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:11:14.692275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:11:14.692275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:11:14.692275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:11:14.692275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:11:14.692275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:11:14.692275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:11:14.692275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 615/1235] 10.222.0.81 () {56 vars in 1102 bytes} [Tue Jun 7 14:11:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:14.698181 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869668d8-dd70-46e6-9095-8d2c59ffa8a9 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:14.745255 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:14.805247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869668d8-dd70-46e6-9095-8d2c59ffa8a9 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:14.840206 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51217c-dfea-4a02-ac9d-60b613e56d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:14.840774 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51217c-dfea-4a02-ac9d-60b613e56d3b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:14.841056 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51217c-dfea-4a02-ac9d-60b613e56d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:14.841327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51217c-dfea-4a02-ac9d-60b613e56d3b None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:14.842073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51217c-dfea-4a02-ac9d-60b613e56d3b None admin] RBAC: Adding query filter params (name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:14.842379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51217c-dfea-4a02-ac9d-60b613e56d3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:14.844446 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51217c-dfea-4a02-ac9d-60b613e56d3b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:14.852944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 621/1236] 10.222.0.81 () {56 vars in 1103 bytes} [Tue Jun 7 14:11:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:14.859332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d079d15-8b24-4642-89b3-c074dbc4479b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:14.958158 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:14.976609 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d079d15-8b24-4642-89b3-c074dbc4479b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:15.013130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d59dc-5270-496d-8d91-1a6b0b6b1195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:15.014088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d59dc-5270-496d-8d91-1a6b0b6b1195 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:15.014468 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d59dc-5270-496d-8d91-1a6b0b6b1195 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:15.015025 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d59dc-5270-496d-8d91-1a6b0b6b1195 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:15.027742 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d59dc-5270-496d-8d91-1a6b0b6b1195 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:15.029435 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d59dc-5270-496d-8d91-1a6b0b6b1195 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:15.050498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f52d59dc-5270-496d-8d91-1a6b0b6b1195 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:11:15.050498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:15.050498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:15.050498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:15.050498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:15.050498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:15.050498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:15.050498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:15.050498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:15.050498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:15.050498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:15.050498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:15.050498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:11:15.050498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:11:15.050498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:11:15.051536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:11:15.051536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:15.051536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:15.051536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:15.051536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:15.051536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:15.051536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:15.051536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:15.051536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:15.051536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:15.051536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:15.051536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:15.051536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:15.051536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:15.051536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:15.051536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:15.051536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:15.052471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:11:15.052471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:11:15.052471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:11:15.052471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:11:15.052471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 616/1237] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:11:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:11:15.058505 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51217c-dfea-4a02-ac9d-60b613e56d3b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:15.166528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51217c-dfea-4a02-ac9d-60b613e56d3b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:15.176561 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:15.203491 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873d861-b536-4123-b77a-18ae3079d1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:15.205203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873d861-b536-4123-b77a-18ae3079d1e3 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:15.205203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873d861-b536-4123-b77a-18ae3079d1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:15.205203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873d861-b536-4123-b77a-18ae3079d1e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:15.205758 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f873d861-b536-4123-b77a-18ae3079d1e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:15.206021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f873d861-b536-4123-b77a-18ae3079d1e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:15.207827 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f873d861-b536-4123-b77a-18ae3079d1e3 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:15.221352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 622/1238] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:15.228180 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d59dc-5270-496d-8d91-1a6b0b6b1195 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:15.345315 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d59dc-5270-496d-8d91-1a6b0b6b1195 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:15.385900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba3a84-a3af-4efc-97d8-e423534c1ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:15.386836 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba3a84-a3af-4efc-97d8-e423534c1ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:15.387125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba3a84-a3af-4efc-97d8-e423534c1ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:15.387585 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba3a84-a3af-4efc-97d8-e423534c1ce2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:15.391558 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:15.398478 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba3a84-a3af-4efc-97d8-e423534c1ce2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:15.400120 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba3a84-a3af-4efc-97d8-e423534c1ce2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:15.417352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-cdba3a84-a3af-4efc-97d8-e423534c1ce2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 14:11:15.417352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:15.417352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:15.417352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:15.417352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:15.417352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:15.417352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:15.417352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:15.417352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:15.417352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:15.417352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:15.417352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:15.417352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:11:15.417352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:11:15.417352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:11:15.417352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:11:15.418562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:15.418562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:15.418562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:11:15.418562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:11:15.418562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:11:15.418562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:11:15.418562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:15.418562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:15.418562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:15.418562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:15.418562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:15.418562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:15.418562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:15.418562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:15.418562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:15.418562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:15.418562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:15.418562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:15.419943 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:15.419943 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:15.419943 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:11:15.419943 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:11:15.419943 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:11:15.419943 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:11:15.419943 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:11:15.419943 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:11:15.419943 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 14:11:15.419943 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:11:15.419943 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 617/1239] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:11:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:15.425404 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873d861-b536-4123-b77a-18ae3079d1e3 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:15.542490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873d861-b536-4123-b77a-18ae3079d1e3 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:15.581663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa82f9-bc6f-43c5-87f6-a88393dae54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:15.582459 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa82f9-bc6f-43c5-87f6-a88393dae54a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:15.582767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa82f9-bc6f-43c5-87f6-a88393dae54a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:15.583069 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa82f9-bc6f-43c5-87f6-a88393dae54a None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:15.584119 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fa82f9-bc6f-43c5-87f6-a88393dae54a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:15.584432 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fa82f9-bc6f-43c5-87f6-a88393dae54a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:15.586014 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fa82f9-bc6f-43c5-87f6-a88393dae54a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:15.606253 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 623/1240] 10.222.0.81 () {56 vars in 1143 bytes} [Tue Jun 7 14:11:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:15.613257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba3a84-a3af-4efc-97d8-e423534c1ce2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:15.737399 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba3a84-a3af-4efc-97d8-e423534c1ce2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:15.773440 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315eaf2c-6faa-4b1e-b926-5a15707176fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:15.774100 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315eaf2c-6faa-4b1e-b926-5a15707176fb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:15.774386 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315eaf2c-6faa-4b1e-b926-5a15707176fb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:15.774689 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315eaf2c-6faa-4b1e-b926-5a15707176fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:15.839118 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:15.845944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315eaf2c-6faa-4b1e-b926-5a15707176fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:15.847552 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315eaf2c-6faa-4b1e-b926-5a15707176fb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:15.958021 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 618/1241] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:11:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:11:15.964875 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa82f9-bc6f-43c5-87f6-a88393dae54a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:15.995850 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:16.084580 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa82f9-bc6f-43c5-87f6-a88393dae54a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:16.119804 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e673efa-56d0-4a8b-9e6c-927be38923b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:16.120528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e673efa-56d0-4a8b-9e6c-927be38923b4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:16.120802 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e673efa-56d0-4a8b-9e6c-927be38923b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:16.121075 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e673efa-56d0-4a8b-9e6c-927be38923b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:16.131742 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e673efa-56d0-4a8b-9e6c-927be38923b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:16.133175 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e673efa-56d0-4a8b-9e6c-927be38923b4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:16.150288 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-3e673efa-56d0-4a8b-9e6c-927be38923b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:11:16.150288 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:16.150288 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:16.150288 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:16.150288 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:16.150288 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:16.150288 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:16.150288 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:16.150288 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:16.150288 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:16.150288 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:16.150288 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:16.150288 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:11:16.150288 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:11:16.150288 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:11:16.151480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:11:16.151480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:16.151480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:16.151480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:16.151480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:16.151480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:16.151480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:16.151480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:16.151480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:16.151480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:16.151480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:16.151480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:16.151480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:16.151480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:16.151480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:16.151480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:16.151480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:16.152919 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:11:16.152919 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:11:16.152919 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:11:16.152919 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:11:16.152919 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 624/1242] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:11:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:16.157846 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315eaf2c-6faa-4b1e-b926-5a15707176fb None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:16.259308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:16.266348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315eaf2c-6faa-4b1e-b926-5a15707176fb None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:16.301000 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a0c7f-f8c7-4ded-8fed-bacbe55f7e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:16.301673 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a0c7f-f8c7-4ded-8fed-bacbe55f7e6b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:16.301970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a0c7f-f8c7-4ded-8fed-bacbe55f7e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:16.302297 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a0c7f-f8c7-4ded-8fed-bacbe55f7e6b None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:16.302989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3a0c7f-f8c7-4ded-8fed-bacbe55f7e6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:16.303315 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3a0c7f-f8c7-4ded-8fed-bacbe55f7e6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:16.304713 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3a0c7f-f8c7-4ded-8fed-bacbe55f7e6b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:16.317124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 619/1243] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:11:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:16.322760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e673efa-56d0-4a8b-9e6c-927be38923b4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:16.408302 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:16.431197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e673efa-56d0-4a8b-9e6c-927be38923b4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:16.466225 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786cf3ef-0820-4c39-8734-0ab0bdb3ea58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:16.467018 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cf3ef-0820-4c39-8734-0ab0bdb3ea58 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:16.467323 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cf3ef-0820-4c39-8734-0ab0bdb3ea58 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:16.467657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cf3ef-0820-4c39-8734-0ab0bdb3ea58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:16.475257 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786cf3ef-0820-4c39-8734-0ab0bdb3ea58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:16.476666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786cf3ef-0820-4c39-8734-0ab0bdb3ea58 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:16.490392 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-786cf3ef-0820-4c39-8734-0ab0bdb3ea58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:11:16.490392 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:16.490392 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:16.490392 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:16.490392 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:16.490392 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:16.490392 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:16.490392 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:16.490392 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:16.490392 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:16.490392 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:16.490392 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:16.490392 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:11:16.490392 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:11:16.490392 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:11:16.491787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:11:16.491787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:16.491787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:16.491787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:16.491787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:16.491787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:16.491787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:16.491787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:16.491787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:16.491787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:16.491787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:16.491787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:16.491787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:16.491787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:16.491787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:16.491787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:16.491787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:16.492747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:11:16.492747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:11:16.492747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:11:16.492747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:11:16.492747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:11:16.492747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:11:16.492747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:11:16.492747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:11:16.492747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 625/1244] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:11:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:16.498059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a0c7f-f8c7-4ded-8fed-bacbe55f7e6b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:16.601961 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a0c7f-f8c7-4ded-8fed-bacbe55f7e6b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:16.637345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d51e5c-d606-4bcd-8519-e241f3ba7ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:16.638205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d51e5c-d606-4bcd-8519-e241f3ba7ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:16.638511 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d51e5c-d606-4bcd-8519-e241f3ba7ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:16.638906 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d51e5c-d606-4bcd-8519-e241f3ba7ed4 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:16.639900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d51e5c-d606-4bcd-8519-e241f3ba7ed4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:16.640206 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d51e5c-d606-4bcd-8519-e241f3ba7ed4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:16.641647 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d51e5c-d606-4bcd-8519-e241f3ba7ed4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:16.654587 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 620/1245] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:11:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:16.662048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786cf3ef-0820-4c39-8734-0ab0bdb3ea58 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:16.768325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786cf3ef-0820-4c39-8734-0ab0bdb3ea58 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:16.804543 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a86b47-bf2a-4cdb-ae2a-13f652de0240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:16.805199 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a86b47-bf2a-4cdb-ae2a-13f652de0240 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:16.805498 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a86b47-bf2a-4cdb-ae2a-13f652de0240 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:16.805756 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a86b47-bf2a-4cdb-ae2a-13f652de0240 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:16.806586 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a86b47-bf2a-4cdb-ae2a-13f652de0240 None admin] RBAC: Adding query filter params (role.id=a72e5464b7b746978805326c224753bc, scope.project.id=f67bad575f8b4a47bee7c314497cb2fe, user.id=ab73cf3febbf4c768cd0116306f16c50) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:16.806897 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a86b47-bf2a-4cdb-ae2a-13f652de0240 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:16.808401 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a86b47-bf2a-4cdb-ae2a-13f652de0240 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:16.815636 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 626/1246] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:11:16 2022] GET /identity/v3/role_assignments?user.id=ab73cf3febbf4c768cd0116306f16c50&scope.project.id=f67bad575f8b4a47bee7c314497cb2fe&role.id=a72e5464b7b746978805326c224753bc => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:11:19.334770 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f0c8d-e896-4f40-898c-900a4cf52dae None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:19.335397 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f0c8d-e896-4f40-898c-900a4cf52dae None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:19.335397 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f0c8d-e896-4f40-898c-900a4cf52dae None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:19.336751 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 621/1247] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:11:19.344865 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641abcbc-3516-4afa-bb09-fe61f8ed7714 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:19.345328 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641abcbc-3516-4afa-bb09-fe61f8ed7714 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:19.345501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641abcbc-3516-4afa-bb09-fe61f8ed7714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:19.351275 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:19.365240 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-641abcbc-3516-4afa-bb09-fe61f8ed7714 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:11:19.366701 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-641abcbc-3516-4afa-bb09-fe61f8ed7714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:11:19.443341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 627/1248] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:11:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:11:19.466136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f0c8d-e896-4f40-898c-900a4cf52dae None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:19.485772 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:19.576102 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f0c8d-e896-4f40-898c-900a4cf52dae None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:19.613301 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb33e29-a9e8-4de3-b01e-45f3d930c601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:19.614252 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb33e29-a9e8-4de3-b01e-45f3d930c601 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:19.614635 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb33e29-a9e8-4de3-b01e-45f3d930c601 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:19.615008 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb33e29-a9e8-4de3-b01e-45f3d930c601 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:19.626274 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb33e29-a9e8-4de3-b01e-45f3d930c601 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:11:19.635691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-6eb33e29-a9e8-4de3-b01e-45f3d930c601 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:11:19.635691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:19.635691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:19.635691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:19.635691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:19.635691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:19.635691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:19.635691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:19.635691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:19.635691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:19.635691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:19.635691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:19.635691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:11:19.635691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:11:19.635691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:11:19.635691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:11:19.636813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:11:19.636813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:11:19.636813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:11:19.636813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:11:19.636813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:19.636813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:19.636813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:19.636813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:19.636813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:19.636813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:19.636813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:19.636813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:19.636813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:19.636813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:19.636813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:19.636813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:19.643394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:19.643394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:19.643394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:19.643394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:19.643394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:11:19.643394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:11:19.643394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:11:19.643394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:11:19.643394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:11:19.643394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:11:19.643394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:11:19.643394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:11:19.643394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 622/1249] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:11:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:19.644276 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641abcbc-3516-4afa-bb09-fe61f8ed7714 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:19.757923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641abcbc-3516-4afa-bb09-fe61f8ed7714 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:19.794472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:19.801170 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce81af-dd01-4b1d-807d-8985788f256b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:19.801859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce81af-dd01-4b1d-807d-8985788f256b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:19.802186 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce81af-dd01-4b1d-807d-8985788f256b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:19.802450 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce81af-dd01-4b1d-807d-8985788f256b None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:19.803470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce81af-dd01-4b1d-807d-8985788f256b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:19.803871 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce81af-dd01-4b1d-807d-8985788f256b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:19.805407 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce81af-dd01-4b1d-807d-8985788f256b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:19.814609 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 628/1250] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:11:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:19.820544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb33e29-a9e8-4de3-b01e-45f3d930c601 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:19.904983 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:19.930880 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb33e29-a9e8-4de3-b01e-45f3d930c601 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:19.970379 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b438c97b-6a56-4495-b9a2-7d303213b03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:19.971178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438c97b-6a56-4495-b9a2-7d303213b03f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:19.971501 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438c97b-6a56-4495-b9a2-7d303213b03f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:19.971929 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438c97b-6a56-4495-b9a2-7d303213b03f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:19.987845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b438c97b-6a56-4495-b9a2-7d303213b03f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:19.989988 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b438c97b-6a56-4495-b9a2-7d303213b03f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:20.018193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-b438c97b-6a56-4495-b9a2-7d303213b03f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:11:20.018193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:20.018193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:20.018193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:20.018193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:20.018193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:20.018193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:20.018193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:20.018193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:20.018193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:20.018193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:20.018193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:20.018193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:11:20.018193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:11:20.018193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:11:20.019156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:11:20.019156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:20.019156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:20.019156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:20.019156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:20.019156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:20.019156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:20.019156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:20.019156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:20.019156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:20.019156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:20.019156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:20.019156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:20.019156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:20.019156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:20.019156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:20.019156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:20.020108 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:11:20.020108 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:11:20.020108 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:11:20.020108 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:11:20.025447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 623/1251] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:11:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:20.026842 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce81af-dd01-4b1d-807d-8985788f256b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:20.137366 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce81af-dd01-4b1d-807d-8985788f256b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:20.180919 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d8c32f-906c-42d6-9a0c-e51dd7fb42ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:20.181591 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8c32f-906c-42d6-9a0c-e51dd7fb42ea None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:20.181876 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8c32f-906c-42d6-9a0c-e51dd7fb42ea None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:20.182155 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8c32f-906c-42d6-9a0c-e51dd7fb42ea None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:20.182884 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d8c32f-906c-42d6-9a0c-e51dd7fb42ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:20.183190 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d8c32f-906c-42d6-9a0c-e51dd7fb42ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:20.184592 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d8c32f-906c-42d6-9a0c-e51dd7fb42ea None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:20.197928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 629/1252] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:20.203940 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b438c97b-6a56-4495-b9a2-7d303213b03f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:20.314527 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b438c97b-6a56-4495-b9a2-7d303213b03f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:20.345465 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:20.355071 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548c97a-48fd-4090-af27-7630349a179a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:20.355924 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548c97a-48fd-4090-af27-7630349a179a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:20.356210 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548c97a-48fd-4090-af27-7630349a179a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:20.356487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548c97a-48fd-4090-af27-7630349a179a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:20.366821 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548c97a-48fd-4090-af27-7630349a179a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:20.368978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548c97a-48fd-4090-af27-7630349a179a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:20.386446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f548c97a-48fd-4090-af27-7630349a179a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 14:11:20.386446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:20.386446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:20.386446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:20.386446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:20.386446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:20.386446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:20.386446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:20.386446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:20.386446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:20.386446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:20.386446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:20.386446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:11:20.386446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:11:20.386446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:11:20.386446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:11:20.387640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:20.387640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:20.387640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:11:20.387640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:11:20.387640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:11:20.387640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:11:20.387640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:20.387640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:20.387640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:20.387640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:20.387640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:20.387640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:20.387640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:20.387640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:20.387640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:20.387640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:20.387640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:20.387640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:20.388990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:20.388990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:20.388990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:11:20.388990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:11:20.388990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:11:20.388990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:11:20.388990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:11:20.388990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:11:20.388990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 14:11:20.388990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:11:20.388990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 624/1253] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:11:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:20.395243 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d8c32f-906c-42d6-9a0c-e51dd7fb42ea None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:20.457050 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:20.503867 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d8c32f-906c-42d6-9a0c-e51dd7fb42ea None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:20.538763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee2dcb-234d-4f6d-b6e8-0add8c6aa7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:20.539529 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee2dcb-234d-4f6d-b6e8-0add8c6aa7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:20.539795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee2dcb-234d-4f6d-b6e8-0add8c6aa7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:20.540109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee2dcb-234d-4f6d-b6e8-0add8c6aa7f7 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:20.540859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee2dcb-234d-4f6d-b6e8-0add8c6aa7f7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:20.541167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee2dcb-234d-4f6d-b6e8-0add8c6aa7f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:20.542585 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee2dcb-234d-4f6d-b6e8-0add8c6aa7f7 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:20.560664 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 630/1254] 10.222.0.81 () {56 vars in 1143 bytes} [Tue Jun 7 14:11:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:20.566861 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548c97a-48fd-4090-af27-7630349a179a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:20.678255 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548c97a-48fd-4090-af27-7630349a179a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:20.715416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8939e-6794-46ca-9dfa-09c240d4e97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:20.716076 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8939e-6794-46ca-9dfa-09c240d4e97f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:20.716366 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8939e-6794-46ca-9dfa-09c240d4e97f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:20.716682 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8939e-6794-46ca-9dfa-09c240d4e97f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:20.775335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:20.789278 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec8939e-6794-46ca-9dfa-09c240d4e97f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:20.790875 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec8939e-6794-46ca-9dfa-09c240d4e97f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:20.904662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 625/1255] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:11:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:11:20.911269 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee2dcb-234d-4f6d-b6e8-0add8c6aa7f7 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:21.037956 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee2dcb-234d-4f6d-b6e8-0add8c6aa7f7 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:21.053130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:21.081227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399588a-bb63-484c-86e4-42f3780b43e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:21.082049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399588a-bb63-484c-86e4-42f3780b43e4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:21.082330 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399588a-bb63-484c-86e4-42f3780b43e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:21.082605 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399588a-bb63-484c-86e4-42f3780b43e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:21.095540 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399588a-bb63-484c-86e4-42f3780b43e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:21.097035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399588a-bb63-484c-86e4-42f3780b43e4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:21.115616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-6399588a-bb63-484c-86e4-42f3780b43e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:11:21.115616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:21.115616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:21.115616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:21.115616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:21.115616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:21.115616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:21.115616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:21.115616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:21.115616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:21.115616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:21.115616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:21.115616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:11:21.115616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:11:21.115616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:11:21.117072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:11:21.117072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:21.117072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:21.117072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:21.117072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:21.117072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:21.117072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:21.117072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:21.117072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:21.117072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:21.117072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:21.117072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:21.117072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:21.117072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:21.117072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:21.117072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:21.117072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:21.118179 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:11:21.118179 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:11:21.118179 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:11:21.118179 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:11:21.118179 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 631/1256] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:11:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:11:21.125423 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8939e-6794-46ca-9dfa-09c240d4e97f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:21.220389 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:21.234785 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8939e-6794-46ca-9dfa-09c240d4e97f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:21.272469 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d7f23-26e9-4e1c-8b0d-60163b86e5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:21.273151 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d7f23-26e9-4e1c-8b0d-60163b86e5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:21.273434 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d7f23-26e9-4e1c-8b0d-60163b86e5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:21.273710 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d7f23-26e9-4e1c-8b0d-60163b86e5a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:21.274443 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d7f23-26e9-4e1c-8b0d-60163b86e5a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:21.274748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d7f23-26e9-4e1c-8b0d-60163b86e5a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:21.276193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d7f23-26e9-4e1c-8b0d-60163b86e5a7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:21.288946 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 626/1257] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:21.294751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399588a-bb63-484c-86e4-42f3780b43e4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:21.400385 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399588a-bb63-484c-86e4-42f3780b43e4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:21.435470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a044c-32cb-4dc6-83c9-8b99abb09094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:21.436240 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a044c-32cb-4dc6-83c9-8b99abb09094 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:21.436528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a044c-32cb-4dc6-83c9-8b99abb09094 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:21.436798 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a044c-32cb-4dc6-83c9-8b99abb09094 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:21.444146 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a044c-32cb-4dc6-83c9-8b99abb09094 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:21.445595 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a044c-32cb-4dc6-83c9-8b99abb09094 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:21.461222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-3e0a044c-32cb-4dc6-83c9-8b99abb09094 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:11:21.461222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:21.461222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:21.461222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:21.461222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:21.461222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:21.461222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:21.461222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:21.461222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:21.461222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:21.461222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:21.461222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:21.461222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:11:21.461222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:11:21.461222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:11:21.462403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:11:21.462403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:21.462403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:21.462403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:21.462403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:21.462403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:21.462403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:21.462403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:21.462403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:21.462403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:21.462403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:21.462403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:21.462403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:21.462403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:21.462403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:21.462403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:21.462403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:21.463369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:11:21.463369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:11:21.463369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:11:21.463369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:11:21.463369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:11:21.463369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:11:21.463369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:11:21.463369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:11:21.463369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 632/1258] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:11:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:11:21.468914 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d7f23-26e9-4e1c-8b0d-60163b86e5a7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:21.572362 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d7f23-26e9-4e1c-8b0d-60163b86e5a7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:21.606676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1bbaed-3925-4fa7-84ac-48d223ee1c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:21.607377 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1bbaed-3925-4fa7-84ac-48d223ee1c9c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:21.607663 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1bbaed-3925-4fa7-84ac-48d223ee1c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:21.607922 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1bbaed-3925-4fa7-84ac-48d223ee1c9c None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:21.609545 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1bbaed-3925-4fa7-84ac-48d223ee1c9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:21.609856 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1bbaed-3925-4fa7-84ac-48d223ee1c9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:21.611305 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1bbaed-3925-4fa7-84ac-48d223ee1c9c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:21.623695 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 627/1259] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:11:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:11:21.629865 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a044c-32cb-4dc6-83c9-8b99abb09094 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:21.656211 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:21.733279 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a044c-32cb-4dc6-83c9-8b99abb09094 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:21.769006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052d32d-27c4-4317-84a3-a4e1d0caffde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:21.769662 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052d32d-27c4-4317-84a3-a4e1d0caffde None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:21.769945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052d32d-27c4-4317-84a3-a4e1d0caffde None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:21.770220 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052d32d-27c4-4317-84a3-a4e1d0caffde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:21.771050 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052d32d-27c4-4317-84a3-a4e1d0caffde None admin] RBAC: Adding query filter params (role.id=82bbe3651b9b4f378fd150d93357b2b1, scope.project.id=f67bad575f8b4a47bee7c314497cb2fe, user.id=ab73cf3febbf4c768cd0116306f16c50) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:21.771393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052d32d-27c4-4317-84a3-a4e1d0caffde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:21.772823 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052d32d-27c4-4317-84a3-a4e1d0caffde None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:21.778178 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 633/1260] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:11:21 2022] GET /identity/v3/role_assignments?user.id=ab73cf3febbf4c768cd0116306f16c50&scope.project.id=f67bad575f8b4a47bee7c314497cb2fe&role.id=82bbe3651b9b4f378fd150d93357b2b1 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:24.004940 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a64c3b-981f-4094-8901-46fa023e0f46 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:24.005502 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a64c3b-981f-4094-8901-46fa023e0f46 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:24.005502 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a64c3b-981f-4094-8901-46fa023e0f46 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:24.006858 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 628/1261] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:11:24.011932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74053b9b-0b4e-4dfa-b5bd-60f1bdfefb59 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:24.012335 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74053b9b-0b4e-4dfa-b5bd-60f1bdfefb59 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:24.012491 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74053b9b-0b4e-4dfa-b5bd-60f1bdfefb59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:24.029032 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-74053b9b-0b4e-4dfa-b5bd-60f1bdfefb59 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:11:24.030912 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-74053b9b-0b4e-4dfa-b5bd-60f1bdfefb59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:11:24.098699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 634/1262] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:11:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:11:24.121809 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a64c3b-981f-4094-8901-46fa023e0f46 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:24.132646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:24.222258 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a64c3b-981f-4094-8901-46fa023e0f46 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:24.255041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f7e2c-7055-4525-9a8e-953b0466595c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:24.255783 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f7e2c-7055-4525-9a8e-953b0466595c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:24.256066 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f7e2c-7055-4525-9a8e-953b0466595c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:24.256338 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f7e2c-7055-4525-9a8e-953b0466595c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:24.266916 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f7e2c-7055-4525-9a8e-953b0466595c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:11:24.274350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-9f9f7e2c-7055-4525-9a8e-953b0466595c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:11:24.274350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:24.274350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:24.274350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:24.274350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:24.274350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:24.274350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:24.274350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:24.274350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:24.274350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:24.274350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:24.274350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:24.274350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:11:24.274350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:11:24.274350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:11:24.274350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:11:24.275476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:11:24.275476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:11:24.275476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:11:24.275476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:11:24.275476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:24.275476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:24.275476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:24.275476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:24.275476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:24.275476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:24.275476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:24.275476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:24.275476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:24.275476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:24.275476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:24.275476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:24.276672 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:24.276672 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:24.276672 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:24.276672 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:24.276672 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:11:24.276672 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:11:24.276672 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:11:24.276672 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:11:24.276672 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:11:24.276672 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:11:24.276672 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:11:24.276672 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:11:24.276672 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 629/1263] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:11:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:24.282223 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74053b9b-0b4e-4dfa-b5bd-60f1bdfefb59 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:24.300075 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:24.379176 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74053b9b-0b4e-4dfa-b5bd-60f1bdfefb59 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:24.413014 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774f029-5c9e-4791-b21c-074ecac7387a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:24.413675 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774f029-5c9e-4791-b21c-074ecac7387a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:24.413957 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774f029-5c9e-4791-b21c-074ecac7387a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:24.414231 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774f029-5c9e-4791-b21c-074ecac7387a None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:24.414968 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a774f029-5c9e-4791-b21c-074ecac7387a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:24.415294 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a774f029-5c9e-4791-b21c-074ecac7387a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:24.416641 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a774f029-5c9e-4791-b21c-074ecac7387a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:24.424665 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 635/1264] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:11:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:24.430463 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f7e2c-7055-4525-9a8e-953b0466595c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:24.499293 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:24.535783 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f7e2c-7055-4525-9a8e-953b0466595c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:24.571233 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea590e2e-1ff6-49f1-82f3-128462ae88ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:24.572049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea590e2e-1ff6-49f1-82f3-128462ae88ba None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:24.572328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea590e2e-1ff6-49f1-82f3-128462ae88ba None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:24.572598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea590e2e-1ff6-49f1-82f3-128462ae88ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:24.584333 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea590e2e-1ff6-49f1-82f3-128462ae88ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:24.585771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea590e2e-1ff6-49f1-82f3-128462ae88ba None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:24.604471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-ea590e2e-1ff6-49f1-82f3-128462ae88ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:11:24.604471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:24.604471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:24.604471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:24.604471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:24.604471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:24.604471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:24.604471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:24.604471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:24.604471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:24.604471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:24.604471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:24.604471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:11:24.604471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:11:24.604471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:11:24.605610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:11:24.605610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:24.605610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:24.605610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:24.605610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:24.605610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:24.605610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:24.605610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:24.605610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:24.605610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:24.605610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:24.605610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:24.605610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:24.605610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:24.605610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:24.605610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:24.605610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:24.606770 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:11:24.606770 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:11:24.606770 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:11:24.606770 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:11:24.606770 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 630/1265] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:11:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:24.611963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774f029-5c9e-4791-b21c-074ecac7387a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:24.703522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:24.724190 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774f029-5c9e-4791-b21c-074ecac7387a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:24.762226 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d2cce-9515-42f2-b085-38bb1a99de8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:24.762888 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d2cce-9515-42f2-b085-38bb1a99de8a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:24.763170 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d2cce-9515-42f2-b085-38bb1a99de8a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:24.763471 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d2cce-9515-42f2-b085-38bb1a99de8a None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:24.764995 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d2cce-9515-42f2-b085-38bb1a99de8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:24.765299 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d2cce-9515-42f2-b085-38bb1a99de8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:24.766975 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d2cce-9515-42f2-b085-38bb1a99de8a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:24.783253 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 636/1266] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:24.789127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea590e2e-1ff6-49f1-82f3-128462ae88ba None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:24.895201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea590e2e-1ff6-49f1-82f3-128462ae88ba None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:24.918646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:24.937010 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b4348-b0ef-400b-a26e-66b2cec2cdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:24.937848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b4348-b0ef-400b-a26e-66b2cec2cdc2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:24.938131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b4348-b0ef-400b-a26e-66b2cec2cdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:24.938412 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b4348-b0ef-400b-a26e-66b2cec2cdc2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:24.947901 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b4348-b0ef-400b-a26e-66b2cec2cdc2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:24.949344 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b4348-b0ef-400b-a26e-66b2cec2cdc2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:24.965236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f66b4348-b0ef-400b-a26e-66b2cec2cdc2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 14:11:24.965236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:24.965236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:24.965236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:24.965236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:24.965236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:24.965236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:24.965236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:24.965236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:24.965236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:24.965236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:24.965236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:24.965236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:11:24.965236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:11:24.965236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:11:24.965236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:11:24.966433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:24.966433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:24.966433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:11:24.966433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:11:24.966433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:11:24.966433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:11:24.966433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:24.966433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:24.966433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:24.966433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:24.966433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:24.966433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:24.966433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:24.966433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:24.966433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:24.966433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:24.966433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:24.966433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:24.967492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:24.967492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:24.967492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:11:24.967492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:11:24.967492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:11:24.967492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:11:24.967492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:11:24.967492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:11:24.967492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 14:11:24.967492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:11:24.967492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 631/1267] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:11:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:24.973637 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d2cce-9515-42f2-b085-38bb1a99de8a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:25.084323 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d2cce-9515-42f2-b085-38bb1a99de8a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:25.122859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51037b-e90c-4c40-ab16-be901d2cf0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:25.123551 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51037b-e90c-4c40-ab16-be901d2cf0fe None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:25.124505 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51037b-e90c-4c40-ab16-be901d2cf0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:25.124892 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51037b-e90c-4c40-ab16-be901d2cf0fe None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:25.125867 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51037b-e90c-4c40-ab16-be901d2cf0fe None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:25.126286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51037b-e90c-4c40-ab16-be901d2cf0fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:25.127809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51037b-e90c-4c40-ab16-be901d2cf0fe None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:25.136700 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:25.145624 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 637/1268] 10.222.0.81 () {56 vars in 1143 bytes} [Tue Jun 7 14:11:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:25.151845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b4348-b0ef-400b-a26e-66b2cec2cdc2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:25.258478 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b4348-b0ef-400b-a26e-66b2cec2cdc2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:25.290957 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac3625-ce1c-4440-9a0e-d670cbf2da71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:25.291765 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac3625-ce1c-4440-9a0e-d670cbf2da71 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:25.292052 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac3625-ce1c-4440-9a0e-d670cbf2da71 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:25.292327 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac3625-ce1c-4440-9a0e-d670cbf2da71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:25.303549 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac3625-ce1c-4440-9a0e-d670cbf2da71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:25.304980 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac3625-ce1c-4440-9a0e-d670cbf2da71 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:25.321610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-5aac3625-ce1c-4440-9a0e-d670cbf2da71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:11:25.321610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:25.321610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:25.321610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:25.321610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:25.321610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:25.321610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:25.321610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:25.321610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:25.321610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:25.321610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:25.321610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:25.321610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:11:25.321610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:11:25.321610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:11:25.322495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:11:25.322495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:25.322495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:25.322495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:25.322495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:25.322495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:25.322495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:25.322495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:25.322495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:25.322495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:25.322495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:25.322495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:25.322495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:25.322495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:25.322495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:25.322495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:25.322495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:25.323485 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:11:25.323485 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:11:25.323485 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:11:25.323485 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:11:25.323485 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 632/1269] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:11:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:25.331494 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51037b-e90c-4c40-ab16-be901d2cf0fe None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:25.437542 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51037b-e90c-4c40-ab16-be901d2cf0fe None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:25.486226 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd398c-cd5d-4082-a375-b5cc542ea63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:25.486874 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd398c-cd5d-4082-a375-b5cc542ea63f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:25.487153 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd398c-cd5d-4082-a375-b5cc542ea63f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:25.487451 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd398c-cd5d-4082-a375-b5cc542ea63f None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:25.488167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd398c-cd5d-4082-a375-b5cc542ea63f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:25.488468 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd398c-cd5d-4082-a375-b5cc542ea63f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:25.489818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd398c-cd5d-4082-a375-b5cc542ea63f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:25.502036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 638/1270] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:11:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:25.507703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac3625-ce1c-4440-9a0e-d670cbf2da71 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:25.539985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:25.610081 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac3625-ce1c-4440-9a0e-d670cbf2da71 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:25.642639 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3039680-8808-4815-aa1c-ac850b9b3bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:25.643443 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3039680-8808-4815-aa1c-ac850b9b3bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:25.643730 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3039680-8808-4815-aa1c-ac850b9b3bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:25.643999 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3039680-8808-4815-aa1c-ac850b9b3bb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:25.650531 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3039680-8808-4815-aa1c-ac850b9b3bb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:25.651976 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3039680-8808-4815-aa1c-ac850b9b3bb2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:25.666234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-c3039680-8808-4815-aa1c-ac850b9b3bb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:11:25.666234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:25.666234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:25.666234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:25.666234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:25.666234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:25.666234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:25.666234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:25.666234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:25.666234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:25.666234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:25.666234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:25.666234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:11:25.666234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:11:25.666234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:11:25.667328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:11:25.667328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:25.667328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:25.667328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:25.667328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:25.667328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:25.667328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:25.667328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:25.667328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:25.667328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:25.667328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:25.667328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:25.667328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:25.667328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:25.667328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:25.667328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:25.667328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:25.668385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:11:25.668385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:11:25.668385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:11:25.668385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:11:25.668385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:11:25.668385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:11:25.668385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:11:25.668385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:11:25.668385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 633/1271] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:11:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:25.673487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd398c-cd5d-4082-a375-b5cc542ea63f None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:25.729933 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:25.770557 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd398c-cd5d-4082-a375-b5cc542ea63f None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:25.802442 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206eb71-4dd6-4175-9918-43ab2db2a247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:25.803097 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206eb71-4dd6-4175-9918-43ab2db2a247 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:25.803409 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206eb71-4dd6-4175-9918-43ab2db2a247 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:25.803687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206eb71-4dd6-4175-9918-43ab2db2a247 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:25.804469 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206eb71-4dd6-4175-9918-43ab2db2a247 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:25.804776 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206eb71-4dd6-4175-9918-43ab2db2a247 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:25.806195 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206eb71-4dd6-4175-9918-43ab2db2a247 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:25.817966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 639/1272] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:11:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:25.823867 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3039680-8808-4815-aa1c-ac850b9b3bb2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:25.914260 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:25.924895 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3039680-8808-4815-aa1c-ac850b9b3bb2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:25.961811 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe5328-ea55-46ca-a7b0-d4059c3b7257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:25.962688 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe5328-ea55-46ca-a7b0-d4059c3b7257 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:25.962967 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe5328-ea55-46ca-a7b0-d4059c3b7257 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:25.963266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe5328-ea55-46ca-a7b0-d4059c3b7257 None admin] PATH_INFO: `/v3/projects/f67bad575f8b4a47bee7c314497cb2fe/users/ab73cf3febbf4c768cd0116306f16c50/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:25.965177 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe5328-ea55-46ca-a7b0-d4059c3b7257 None admin] RBAC: Authorizing `identity:create_grant(project_id=f67bad575f8b4a47bee7c314497cb2fe, user_id=ab73cf3febbf4c768cd0116306f16c50, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:25.966590 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe5328-ea55-46ca-a7b0-d4059c3b7257 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:25.979357 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 634/1273] 10.222.0.81 () {58 vars in 1439 bytes} [Tue Jun 7 14:11:25 2022] PUT /identity/v3/projects/f67bad575f8b4a47bee7c314497cb2fe/users/ab73cf3febbf4c768cd0116306f16c50/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:11:28.195989 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273b438-e4dd-4bfb-b1af-9a245a30f16f None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:28.196439 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273b438-e4dd-4bfb-b1af-9a245a30f16f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:28.196528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273b438-e4dd-4bfb-b1af-9a245a30f16f None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:28.197886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 640/1274] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:11:28.203489 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cc390-42ad-4d45-adb4-5af9fd7fa25f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:28.203875 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cc390-42ad-4d45-adb4-5af9fd7fa25f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:28.204055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cc390-42ad-4d45-adb4-5af9fd7fa25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:28.220907 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-649cc390-42ad-4d45-adb4-5af9fd7fa25f None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:11:28.222079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-649cc390-42ad-4d45-adb4-5af9fd7fa25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:11:28.286430 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 635/1275] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:11:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:11:28.309887 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273b438-e4dd-4bfb-b1af-9a245a30f16f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:28.413767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273b438-e4dd-4bfb-b1af-9a245a30f16f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:28.430316 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:28.447255 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cebfa3-7695-472c-80fa-f556b4b77049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:28.448039 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cebfa3-7695-472c-80fa-f556b4b77049 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:28.448318 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cebfa3-7695-472c-80fa-f556b4b77049 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:28.448592 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cebfa3-7695-472c-80fa-f556b4b77049 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:28.459140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cebfa3-7695-472c-80fa-f556b4b77049 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:11:28.466599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-19cebfa3-7695-472c-80fa-f556b4b77049 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:11:28.466599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:28.466599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:28.466599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:28.466599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:28.466599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:28.466599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:28.466599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:28.466599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:28.466599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:28.466599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:28.466599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:28.466599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:11:28.466599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:11:28.466599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:11:28.466599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:11:28.467932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:11:28.467932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:11:28.467932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:11:28.467932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:11:28.467932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:28.467932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:28.467932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:28.467932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:28.467932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:28.467932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:28.467932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:28.467932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:28.467932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:28.467932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:28.467932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:28.467932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:28.468864 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:28.468864 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:28.468864 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:28.468864 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:28.468864 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:11:28.468864 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:11:28.468864 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:11:28.468864 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:11:28.468864 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:11:28.468864 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:11:28.468864 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:11:28.468864 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:11:28.468864 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 641/1276] 10.222.0.81 () {56 vars in 1096 bytes} [Tue Jun 7 14:11:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:28.474001 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649cc390-42ad-4d45-adb4-5af9fd7fa25f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:28.555240 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:28.574024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649cc390-42ad-4d45-adb4-5af9fd7fa25f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:28.606363 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6cbea-0d2a-482f-a322-c95b32bf164f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:28.607020 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6cbea-0d2a-482f-a322-c95b32bf164f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:28.607327 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6cbea-0d2a-482f-a322-c95b32bf164f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:28.607599 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6cbea-0d2a-482f-a322-c95b32bf164f None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:28.608688 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6cbea-0d2a-482f-a322-c95b32bf164f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:28.608995 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6cbea-0d2a-482f-a322-c95b32bf164f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:28.610363 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6cbea-0d2a-482f-a322-c95b32bf164f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:28.618817 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 636/1277] 10.222.0.81 () {56 vars in 1099 bytes} [Tue Jun 7 14:11:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:28.625321 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cebfa3-7695-472c-80fa-f556b4b77049 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:28.722541 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cebfa3-7695-472c-80fa-f556b4b77049 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:28.755977 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2150439-54f4-4aaa-80ae-bf3481926fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:28.756754 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2150439-54f4-4aaa-80ae-bf3481926fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:28.757038 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2150439-54f4-4aaa-80ae-bf3481926fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:28.757312 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2150439-54f4-4aaa-80ae-bf3481926fe1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:28.766983 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2150439-54f4-4aaa-80ae-bf3481926fe1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:28.768529 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2150439-54f4-4aaa-80ae-bf3481926fe1 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:28.785480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-f2150439-54f4-4aaa-80ae-bf3481926fe1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:11:28.785480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:28.785480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:28.785480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:28.785480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:28.785480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:28.785480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:28.785480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:28.785480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:28.785480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:28.785480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:28.785480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:28.785480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:11:28.785480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:11:28.785480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:11:28.786500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:11:28.786500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:28.786500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:28.786500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:28.786500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:28.786500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:28.786500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:28.786500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:28.786500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:28.786500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:28.786500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:28.786500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:28.786500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:28.786500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:28.786500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:28.786500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:28.786500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:28.792172 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:11:28.792172 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:11:28.792172 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:11:28.792172 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:11:28.792172 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 642/1278] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:11:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:28.792580 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6cbea-0d2a-482f-a322-c95b32bf164f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:28.890542 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6cbea-0d2a-482f-a322-c95b32bf164f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:28.926299 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b6278-08d3-472b-9e43-ca298c74d882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:28.927352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b6278-08d3-472b-9e43-ca298c74d882 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:28.927703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b6278-08d3-472b-9e43-ca298c74d882 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:28.928019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b6278-08d3-472b-9e43-ca298c74d882 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:28.928921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b6278-08d3-472b-9e43-ca298c74d882 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:28.929266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b6278-08d3-472b-9e43-ca298c74d882 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:28.930664 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b6278-08d3-472b-9e43-ca298c74d882 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:28.943461 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 637/1279] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:11:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:28.949574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2150439-54f4-4aaa-80ae-bf3481926fe1 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:28.985004 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:29.053656 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2150439-54f4-4aaa-80ae-bf3481926fe1 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:29.086492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29beea69-3eff-4ec4-80ff-9d590cc3d5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:29.087311 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29beea69-3eff-4ec4-80ff-9d590cc3d5df None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:29.087596 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29beea69-3eff-4ec4-80ff-9d590cc3d5df None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:29.087868 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29beea69-3eff-4ec4-80ff-9d590cc3d5df None admin] PATH_INFO: `/v3/users/placement` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:29.095336 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29beea69-3eff-4ec4-80ff-9d590cc3d5df None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:29.096771 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29beea69-3eff-4ec4-80ff-9d590cc3d5df None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:29.111074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-29beea69-3eff-4ec4-80ff-9d590cc3d5df None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 14:11:29.111074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:29.111074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:29.111074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:29.111074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:29.111074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:29.111074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:29.111074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:29.111074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:29.111074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:29.111074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:29.111074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:29.111074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:11:29.111074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:11:29.111074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:11:29.111074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:11:29.112245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:29.112245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:29.112245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:11:29.112245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:11:29.112245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:11:29.112245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:11:29.112245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:29.112245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:29.112245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:29.112245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:29.112245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:29.112245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:29.112245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:29.112245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:29.112245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:29.112245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:29.112245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:29.112245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:29.113702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:29.113702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:29.113702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:11:29.113702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:11:29.113702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:11:29.113702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:11:29.113702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:11:29.113702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:11:29.113702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 14:11:29.113702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:11:29.113702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 643/1280] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:11:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:29.118336 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b6278-08d3-472b-9e43-ca298c74d882 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:29.122090 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:29.217096 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b6278-08d3-472b-9e43-ca298c74d882 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:29.250440 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfedce9-87d7-411d-b143-c2bfb45fa62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:29.251034 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfedce9-87d7-411d-b143-c2bfb45fa62f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:29.251345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfedce9-87d7-411d-b143-c2bfb45fa62f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:29.251625 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfedce9-87d7-411d-b143-c2bfb45fa62f None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:29.252456 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfedce9-87d7-411d-b143-c2bfb45fa62f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:29.252767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfedce9-87d7-411d-b143-c2bfb45fa62f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:29.254197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfedce9-87d7-411d-b143-c2bfb45fa62f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:29.272022 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 638/1281] 10.222.0.81 () {56 vars in 1143 bytes} [Tue Jun 7 14:11:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:29.279650 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29beea69-3eff-4ec4-80ff-9d590cc3d5df None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:29.366280 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:29.377139 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29beea69-3eff-4ec4-80ff-9d590cc3d5df None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:29.410066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5d295-0d0a-4dc0-a69a-a1b75cb109f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:29.410703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5d295-0d0a-4dc0-a69a-a1b75cb109f9 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:29.410986 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5d295-0d0a-4dc0-a69a-a1b75cb109f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:29.411293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5d295-0d0a-4dc0-a69a-a1b75cb109f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:29.475594 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5d295-0d0a-4dc0-a69a-a1b75cb109f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:29.477046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5d295-0d0a-4dc0-a69a-a1b75cb109f9 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:29.564192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:29.574815 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 644/1282] 10.222.0.81 () {58 vars in 1282 bytes} [Tue Jun 7 14:11:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:11:29.580679 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfedce9-87d7-411d-b143-c2bfb45fa62f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:29.640937 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:29.678420 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfedce9-87d7-411d-b143-c2bfb45fa62f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:29.711342 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc008bb-e45d-4b69-bd08-b9b07e5f7ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:29.712131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc008bb-e45d-4b69-bd08-b9b07e5f7ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:29.712413 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc008bb-e45d-4b69-bd08-b9b07e5f7ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:29.712688 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc008bb-e45d-4b69-bd08-b9b07e5f7ed8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:29.722589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc008bb-e45d-4b69-bd08-b9b07e5f7ed8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:29.724051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc008bb-e45d-4b69-bd08-b9b07e5f7ed8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:29.741787 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-2bc008bb-e45d-4b69-bd08-b9b07e5f7ed8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:11:29.741787 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:29.741787 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:29.741787 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:29.741787 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:29.741787 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:29.741787 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:29.741787 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:29.741787 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:29.741787 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:29.741787 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:29.741787 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:29.741787 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:11:29.741787 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:11:29.741787 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:11:29.742854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:11:29.742854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:29.742854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:29.742854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:29.742854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:29.742854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:29.742854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:29.742854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:29.742854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:29.742854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:29.742854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:29.742854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:29.742854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:29.742854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:29.742854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:29.742854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:29.742854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:29.744169 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:11:29.744169 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:11:29.744169 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:11:29.744169 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:11:29.744169 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 639/1283] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:11:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:29.749253 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5d295-0d0a-4dc0-a69a-a1b75cb109f9 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:29.843582 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5d295-0d0a-4dc0-a69a-a1b75cb109f9 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:29.877070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade5510-2d10-40e1-aca5-8e79bd54d126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:29.877707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade5510-2d10-40e1-aca5-8e79bd54d126 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:29.877982 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade5510-2d10-40e1-aca5-8e79bd54d126 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:29.878251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade5510-2d10-40e1-aca5-8e79bd54d126 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:29.878958 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade5510-2d10-40e1-aca5-8e79bd54d126 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:29.879279 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade5510-2d10-40e1-aca5-8e79bd54d126 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:29.880620 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade5510-2d10-40e1-aca5-8e79bd54d126 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:29.891589 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 645/1284] 10.222.0.81 () {56 vars in 1109 bytes} [Tue Jun 7 14:11:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:29.897230 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc008bb-e45d-4b69-bd08-b9b07e5f7ed8 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:29.996933 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc008bb-e45d-4b69-bd08-b9b07e5f7ed8 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:30.010398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:30.032714 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86826979-4284-42d0-ae36-6a985ac7be28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:30.033496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86826979-4284-42d0-ae36-6a985ac7be28 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:30.033775 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86826979-4284-42d0-ae36-6a985ac7be28 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:30.034046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86826979-4284-42d0-ae36-6a985ac7be28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:30.040471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86826979-4284-42d0-ae36-6a985ac7be28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:30.041902 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86826979-4284-42d0-ae36-6a985ac7be28 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:30.056030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-86826979-4284-42d0-ae36-6a985ac7be28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:11:30.056030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:30.056030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:30.056030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:30.056030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:30.056030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:30.056030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:30.056030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:30.056030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:30.056030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:30.056030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:30.056030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:30.056030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:11:30.056030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:11:30.056030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:11:30.057104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:11:30.057104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:30.057104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:30.057104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:30.057104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:30.057104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:30.057104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:30.057104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:30.057104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:30.057104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:30.057104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:30.057104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:30.057104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:30.057104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:30.057104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:30.057104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:30.057104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:30.058213 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:11:30.058213 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:11:30.058213 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:11:30.058213 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:11:30.058213 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:11:30.058213 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:11:30.058213 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:11:30.058213 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Jun 07 14:11:30.058213 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 640/1285] 10.222.0.81 () {56 vars in 1111 bytes} [Tue Jun 7 14:11:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:30.063067 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade5510-2d10-40e1-aca5-8e79bd54d126 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:30.112717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:30.162428 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade5510-2d10-40e1-aca5-8e79bd54d126 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:30.194927 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b38c8-f953-4595-83ac-f9d1b8dd295b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:30.195600 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b38c8-f953-4595-83ac-f9d1b8dd295b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:30.195879 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b38c8-f953-4595-83ac-f9d1b8dd295b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:30.196152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b38c8-f953-4595-83ac-f9d1b8dd295b None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:30.196926 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b38c8-f953-4595-83ac-f9d1b8dd295b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:30.197228 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b38c8-f953-4595-83ac-f9d1b8dd295b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:30.198635 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b38c8-f953-4595-83ac-f9d1b8dd295b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:30.210976 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 646/1286] 10.222.0.81 () {56 vars in 1148 bytes} [Tue Jun 7 14:11:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:30.217042 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86826979-4284-42d0-ae36-6a985ac7be28 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:30.320514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86826979-4284-42d0-ae36-6a985ac7be28 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:30.353661 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee784a-6150-46d3-ac61-9122be5c75fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:30.354305 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee784a-6150-46d3-ac61-9122be5c75fd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:30.354585 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee784a-6150-46d3-ac61-9122be5c75fd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:30.354853 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee784a-6150-46d3-ac61-9122be5c75fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:30.355698 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee784a-6150-46d3-ac61-9122be5c75fd None admin] RBAC: Adding query filter params (role.id=82bbe3651b9b4f378fd150d93357b2b1, scope.project.id=f67bad575f8b4a47bee7c314497cb2fe, user.id=ab73cf3febbf4c768cd0116306f16c50) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:30.356002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee784a-6150-46d3-ac61-9122be5c75fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:30.357425 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee784a-6150-46d3-ac61-9122be5c75fd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:30.363603 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 641/1287] 10.222.0.81 () {56 vars in 1374 bytes} [Tue Jun 7 14:11:30 2022] GET /identity/v3/role_assignments?user.id=ab73cf3febbf4c768cd0116306f16c50&scope.project.id=f67bad575f8b4a47bee7c314497cb2fe&role.id=82bbe3651b9b4f378fd150d93357b2b1 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:32.618557 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108fcca-4f46-42cb-812c-bf4d123d4349 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:32.619085 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108fcca-4f46-42cb-812c-bf4d123d4349 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:32.619085 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108fcca-4f46-42cb-812c-bf4d123d4349 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:32.620493 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 647/1288] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:11:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:11:32.626413 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067fc80-8f44-42e2-8786-b38d86759034 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:32.626791 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067fc80-8f44-42e2-8786-b38d86759034 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:32.626970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067fc80-8f44-42e2-8786-b38d86759034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:32.643430 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9067fc80-8f44-42e2-8786-b38d86759034 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:11:32.644608 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9067fc80-8f44-42e2-8786-b38d86759034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:11:32.673726 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:32.710253 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 642/1289] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:11:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:11:32.732459 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108fcca-4f46-42cb-812c-bf4d123d4349 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:32.831232 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108fcca-4f46-42cb-812c-bf4d123d4349 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:32.841373 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:32.864630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc83830-0cbe-409e-a677-6336bdaafe46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:32.865420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc83830-0cbe-409e-a677-6336bdaafe46 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:32.865954 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc83830-0cbe-409e-a677-6336bdaafe46 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:32.866228 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc83830-0cbe-409e-a677-6336bdaafe46 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:32.866873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc83830-0cbe-409e-a677-6336bdaafe46 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:32.868257 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc83830-0cbe-409e-a677-6336bdaafe46 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:32.880546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-bfc83830-0cbe-409e-a677-6336bdaafe46 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 14:11:32.880546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:32.880546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:11:32.880546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:11:32.880546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:11:32.880546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:11:32.880546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:11:32.880546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:11:32.880546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 14:11:32.880546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:11:32.880546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:11:32.880546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:11:32.880546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:32.880546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:32.880546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:32.880546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:32.880546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:32.881853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:32.881853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:32.881853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:32.881853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:32.881853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:32.881853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:32.881853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:11:32.881853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:11:32.881853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:11:32.881853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:11:32.881853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:32.881853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:32.881853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:32.881853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:32.881853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:32.881853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:32.882941 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:32.882941 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:32.882941 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:32.882941 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:32.882941 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:32.882941 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:32.882941 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:32.882941 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:32.882941 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:32.882941 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:32.882941 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:11:32.882941 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:11:32.882941 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 14:11:32.882941 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:11:32.882941 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 648/1290] 10.222.0.81 () {56 vars in 1117 bytes} [Tue Jun 7 14:11:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:32.887697 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067fc80-8f44-42e2-8786-b38d86759034 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:33.002349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067fc80-8f44-42e2-8786-b38d86759034 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:33.035418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed18f0-902a-4850-bd31-9c974789a3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:33.036081 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed18f0-902a-4850-bd31-9c974789a3a6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:33.036367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed18f0-902a-4850-bd31-9c974789a3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:33.036644 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed18f0-902a-4850-bd31-9c974789a3a6 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:33.037346 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ed18f0-902a-4850-bd31-9c974789a3a6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:33.037658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ed18f0-902a-4850-bd31-9c974789a3a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:33.039023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ed18f0-902a-4850-bd31-9c974789a3a6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:33.043768 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 643/1291] 10.222.0.81 () {56 vars in 1116 bytes} [Tue Jun 7 14:11:32 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:33.049812 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc83830-0cbe-409e-a677-6336bdaafe46 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:33.147709 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc83830-0cbe-409e-a677-6336bdaafe46 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:33.181022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41760b-ff51-4534-9b49-b12863ceb6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:33.181685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41760b-ff51-4534-9b49-b12863ceb6fb None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:33.181968 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41760b-ff51-4534-9b49-b12863ceb6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:33.182245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41760b-ff51-4534-9b49-b12863ceb6fb None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:33.182950 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41760b-ff51-4534-9b49-b12863ceb6fb None admin] RBAC: Adding query filter params (type=placement) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:33.183286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41760b-ff51-4534-9b49-b12863ceb6fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:33.184642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41760b-ff51-4534-9b49-b12863ceb6fb None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:33.189798 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 649/1292] 10.222.0.81 () {56 vars in 1116 bytes} [Tue Jun 7 14:11:33 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:35.427645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73ae1c-eaad-4d63-9713-84ba9e926394 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:35.428166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73ae1c-eaad-4d63-9713-84ba9e926394 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:35.428166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73ae1c-eaad-4d63-9713-84ba9e926394 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:35.429479 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 644/1293] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:11:35.434507 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc9c76-512d-4093-970b-a5c0a0fa30ba None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:35.434923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc9c76-512d-4093-970b-a5c0a0fa30ba None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:35.435058 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc9c76-512d-4093-970b-a5c0a0fa30ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:35.452289 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-b1dc9c76-512d-4093-970b-a5c0a0fa30ba None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:11:35.453463 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b1dc9c76-512d-4093-970b-a5c0a0fa30ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:11:35.482388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:35.522362 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 650/1294] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:11:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:11:35.545425 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73ae1c-eaad-4d63-9713-84ba9e926394 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:35.575372 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:35.645561 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73ae1c-eaad-4d63-9713-84ba9e926394 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:35.678633 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4549e-7dc2-42e9-bef7-1194add4e1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:35.679251 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4549e-7dc2-42e9-bef7-1194add4e1f3 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:35.679540 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4549e-7dc2-42e9-bef7-1194add4e1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:35.679816 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4549e-7dc2-42e9-bef7-1194add4e1f3 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:35.680543 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4549e-7dc2-42e9-bef7-1194add4e1f3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:35.681892 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4549e-7dc2-42e9-bef7-1194add4e1f3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:35.694337 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 645/1295] 10.222.0.81 () {60 vars in 1141 bytes} [Tue Jun 7 14:11:35 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:11:37.937660 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560b7c7-7205-4fbd-8f70-04abedd0c740 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:37.938309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560b7c7-7205-4fbd-8f70-04abedd0c740 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:37.938309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560b7c7-7205-4fbd-8f70-04abedd0c740 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:37.939615 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 651/1296] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:11:37.945017 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4978e8a-2aad-4c6e-bd2a-702734ce3266 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:37.945492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4978e8a-2aad-4c6e-bd2a-702734ce3266 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:37.945601 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4978e8a-2aad-4c6e-bd2a-702734ce3266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:37.963695 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b4978e8a-2aad-4c6e-bd2a-702734ce3266 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:11:37.964885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b4978e8a-2aad-4c6e-bd2a-702734ce3266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:11:38.035932 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 646/1297] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:11:37 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:11:38.058371 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560b7c7-7205-4fbd-8f70-04abedd0c740 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:38.157235 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560b7c7-7205-4fbd-8f70-04abedd0c740 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:38.198012 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ac49c-697f-4e3d-acf1-edb015a537d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:38.198738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ac49c-697f-4e3d-acf1-edb015a537d0 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:38.199021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ac49c-697f-4e3d-acf1-edb015a537d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:38.199325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ac49c-697f-4e3d-acf1-edb015a537d0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:38.199982 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ac49c-697f-4e3d-acf1-edb015a537d0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:38.201456 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ac49c-697f-4e3d-acf1-edb015a537d0 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:38.213167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-ec7ac49c-697f-4e3d-acf1-edb015a537d0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 14:11:38.213167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:38.213167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:11:38.213167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:11:38.213167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:11:38.213167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:11:38.213167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:11:38.213167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:11:38.213167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 14:11:38.213167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:11:38.213167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:11:38.213167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:11:38.213167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:38.213167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:38.213167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:38.213167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:38.213167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:38.214388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:38.214388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:38.214388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:38.214388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:38.214388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:38.214388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:38.214388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:11:38.214388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:11:38.214388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:11:38.214388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:11:38.214388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:38.214388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:38.214388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:38.214388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:38.214388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:38.214388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:38.215687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:38.215687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:38.215687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:38.215687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:38.215687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:38.215687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:38.215687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:38.215687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:38.215687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:38.215687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:38.215687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:11:38.215687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:11:38.215687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 14:11:38.215687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:11:38.215687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 652/1298] 10.222.0.81 () {56 vars in 1117 bytes} [Tue Jun 7 14:11:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:38.220991 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4978e8a-2aad-4c6e-bd2a-702734ce3266 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:38.246826 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:38.324858 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4978e8a-2aad-4c6e-bd2a-702734ce3266 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:38.358035 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d45be71-3f72-4d0a-a773-f882f0fb33b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:38.358698 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d45be71-3f72-4d0a-a773-f882f0fb33b3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:38.358983 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d45be71-3f72-4d0a-a773-f882f0fb33b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:38.359282 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d45be71-3f72-4d0a-a773-f882f0fb33b3 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:38.359994 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d45be71-3f72-4d0a-a773-f882f0fb33b3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:38.360298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d45be71-3f72-4d0a-a773-f882f0fb33b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:38.361658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d45be71-3f72-4d0a-a773-f882f0fb33b3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:38.366923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 647/1299] 10.222.0.81 () {56 vars in 1116 bytes} [Tue Jun 7 14:11:38 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:38.372992 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ac49c-697f-4e3d-acf1-edb015a537d0 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:38.405675 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:38.472135 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ac49c-697f-4e3d-acf1-edb015a537d0 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:38.504221 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df4d36-9c8c-480c-9e3e-1debf799c039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:38.504865 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df4d36-9c8c-480c-9e3e-1debf799c039 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:38.505149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df4d36-9c8c-480c-9e3e-1debf799c039 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:38.505427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df4d36-9c8c-480c-9e3e-1debf799c039 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:38.506210 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df4d36-9c8c-480c-9e3e-1debf799c039 None admin] RBAC: Adding query filter params (interface=public, service_id=f2c0eb2a57d94c61948e52e94405a219, region_id=RegionOne) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:38.506526 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df4d36-9c8c-480c-9e3e-1debf799c039 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:38.507912 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df4d36-9c8c-480c-9e3e-1debf799c039 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:38.512808 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 653/1300] 10.222.0.81 () {56 vars in 1251 bytes} [Tue Jun 7 14:11:38 2022] GET /identity/v3/endpoints?service_id=f2c0eb2a57d94c61948e52e94405a219&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:38.518894 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d45be71-3f72-4d0a-a773-f882f0fb33b3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:38.596442 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:38.614304 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d45be71-3f72-4d0a-a773-f882f0fb33b3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:38.647631 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7bed2-30b2-442f-84a4-eef0d9d6d7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:38.648282 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7bed2-30b2-442f-84a4-eef0d9d6d7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:38.648561 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7bed2-30b2-442f-84a4-eef0d9d6d7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:38.648832 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7bed2-30b2-442f-84a4-eef0d9d6d7d9 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:38.649512 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7bed2-30b2-442f-84a4-eef0d9d6d7d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:38.650878 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7bed2-30b2-442f-84a4-eef0d9d6d7d9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:38.658239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 648/1301] 10.222.0.81 () {56 vars in 1087 bytes} [Tue Jun 7 14:11:38 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:11:40.880321 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765851fd-374e-49ca-9faa-95a5cd996ddd None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:40.880853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765851fd-374e-49ca-9faa-95a5cd996ddd None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:40.880853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765851fd-374e-49ca-9faa-95a5cd996ddd None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:40.882219 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 654/1302] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:11:40.887476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4740c9-d528-4de7-b16e-9bac62b589e1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:40.887886 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4740c9-d528-4de7-b16e-9bac62b589e1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:40.888034 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4740c9-d528-4de7-b16e-9bac62b589e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:40.905079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-db4740c9-d528-4de7-b16e-9bac62b589e1 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:11:40.906268 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-db4740c9-d528-4de7-b16e-9bac62b589e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:11:40.969578 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 649/1303] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:11:40 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:11:40.993015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765851fd-374e-49ca-9faa-95a5cd996ddd None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:41.085381 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:41.093206 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765851fd-374e-49ca-9faa-95a5cd996ddd None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:41.126100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf628d-d548-42aa-a3ff-f663e3c9f01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:41.126872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf628d-d548-42aa-a3ff-f663e3c9f01a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:41.127151 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf628d-d548-42aa-a3ff-f663e3c9f01a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:41.127452 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf628d-d548-42aa-a3ff-f663e3c9f01a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:41.128091 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf628d-d548-42aa-a3ff-f663e3c9f01a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:41.129428 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf628d-d548-42aa-a3ff-f663e3c9f01a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:41.141109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-d4cf628d-d548-42aa-a3ff-f663e3c9f01a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 14:11:41.141109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:41.141109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:11:41.141109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:11:41.141109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:11:41.141109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:11:41.141109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:11:41.141109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:11:41.141109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 14:11:41.141109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:11:41.141109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:11:41.141109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:11:41.141109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:11:41.141109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:11:41.141109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:11:41.141109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:11:41.141109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:11:41.142325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:11:41.142325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:11:41.142325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:11:41.142325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:11:41.142325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:11:41.142325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:11:41.142325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:11:41.142325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:11:41.142325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:11:41.142325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:11:41.142325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:11:41.142325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:11:41.142325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:11:41.142325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:11:41.142325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:11:41.142325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:11:41.143352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:11:41.143352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( Jun 07 14:11:41.143352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:11:41.143352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:11:41.143352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:11:41.143352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:11:41.143352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:11:41.143352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:11:41.143352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:11:41.143352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:11:41.143352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:11:41.143352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:11:41.143352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 14:11:41.143352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Jun 07 14:11:41.143352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 655/1304] 10.222.0.81 () {56 vars in 1117 bytes} [Tue Jun 7 14:11:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:11:41.148358 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4740c9-d528-4de7-b16e-9bac62b589e1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:41.222754 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:41.246066 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4740c9-d528-4de7-b16e-9bac62b589e1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:41.277765 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6295d33-d747-439a-b71c-8396c71ade2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:41.278425 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6295d33-d747-439a-b71c-8396c71ade2f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:41.278709 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6295d33-d747-439a-b71c-8396c71ade2f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:41.278985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6295d33-d747-439a-b71c-8396c71ade2f None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:41.280332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6295d33-d747-439a-b71c-8396c71ade2f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:11:41.280636 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6295d33-d747-439a-b71c-8396c71ade2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:41.282026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6295d33-d747-439a-b71c-8396c71ade2f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:41.287025 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 650/1305] 10.222.0.81 () {56 vars in 1116 bytes} [Tue Jun 7 14:11:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:11:41.292825 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf628d-d548-42aa-a3ff-f663e3c9f01a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:11:41.390355 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf628d-d548-42aa-a3ff-f663e3c9f01a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:11:41.422682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f575c769-fd8a-408e-a5a9-8d1e4dd14088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:11:41.423277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575c769-fd8a-408e-a5a9-8d1e4dd14088 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:11:41.423559 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575c769-fd8a-408e-a5a9-8d1e4dd14088 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:11:41.423830 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575c769-fd8a-408e-a5a9-8d1e4dd14088 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:11:41.424675 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f575c769-fd8a-408e-a5a9-8d1e4dd14088 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:11:41.426020 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f575c769-fd8a-408e-a5a9-8d1e4dd14088 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:11:41.443556 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 656/1306] 10.222.0.81 () {60 vars in 1144 bytes} [Tue Jun 7 14:11:41 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:11:51.284139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=30 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:11:51.439377 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:12:46.004773 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac581dd-51f9-4e22-9426-13276cf58b10 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:12:46.006001 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac581dd-51f9-4e22-9426-13276cf58b10 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:12:46.006663 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac581dd-51f9-4e22-9426-13276cf58b10 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:12:46.008898 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 651/1307] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:12:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:12:46.014847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef950126-b16e-469e-b894-be17882b7a90 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:12:46.015566 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef950126-b16e-469e-b894-be17882b7a90 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:12:46.015566 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef950126-b16e-469e-b894-be17882b7a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:12:46.126483 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-ef950126-b16e-469e-b894-be17882b7a90 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:12:46.129681 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ef950126-b16e-469e-b894-be17882b7a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:12:46.275032 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 657/1308] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:12:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:12:46.326001 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a95ed-4ac0-41bf-9652-2f076ca4b64a None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:12:46.326188 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a95ed-4ac0-41bf-9652-2f076ca4b64a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:12:46.326462 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a95ed-4ac0-41bf-9652-2f076ca4b64a None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:12:46.327910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 652/1309] 10.222.0.81 () {54 vars in 964 bytes} [Tue Jun 7 14:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:12:46.334341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0385dc-ed9b-405e-955a-0d6935c83b3b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:12:46.334796 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0385dc-ed9b-405e-955a-0d6935c83b3b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:12:46.335328 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0385dc-ed9b-405e-955a-0d6935c83b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:12:46.421859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6a0385dc-ed9b-405e-955a-0d6935c83b3b None None] MFA Rules not processed for user `a6a08eaecbcd41228571739d8387882a`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:12:46.423180 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6a0385dc-ed9b-405e-955a-0d6935c83b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:12:46.444677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:12:46.504560 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 658/1310] 10.222.0.81 () {58 vars in 1061 bytes} [Tue Jun 7 14:12:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:12:46.514189 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a95ed-4ac0-41bf-9652-2f076ca4b64a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:12:46.516082 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-413a95ed-4ac0-41bf-9652-2f076ca4b64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:12:46.627110 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a95ed-4ac0-41bf-9652-2f076ca4b64a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:12:46.706553 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77eecdb-4b7c-4bb8-90f0-78e56d3ba427 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:12:46.707400 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77eecdb-4b7c-4bb8-90f0-78e56d3ba427 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:12:46.707782 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77eecdb-4b7c-4bb8-90f0-78e56d3ba427 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:12:46.708184 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77eecdb-4b7c-4bb8-90f0-78e56d3ba427 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:12:46.709921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d77eecdb-4b7c-4bb8-90f0-78e56d3ba427 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:12:46.767696 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:12:46.827984 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77eecdb-4b7c-4bb8-90f0-78e56d3ba427 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:12:46.829575 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77eecdb-4b7c-4bb8-90f0-78e56d3ba427 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:12:46.848891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 653/1311] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:12:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:12:49.470884 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ca52a-3b2e-4b74-b7d3-05f33269ff36 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:12:49.471443 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ca52a-3b2e-4b74-b7d3-05f33269ff36 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:12:49.471523 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ca52a-3b2e-4b74-b7d3-05f33269ff36 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:12:49.472857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 659/1312] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:12:49.478379 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e2484-ab8e-46c0-86e5-3f747937e973 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:12:49.478808 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e2484-ab8e-46c0-86e5-3f747937e973 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:12:49.478947 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e2484-ab8e-46c0-86e5-3f747937e973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:12:49.575981 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-695e2484-ab8e-46c0-86e5-3f747937e973 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:12:49.577101 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-695e2484-ab8e-46c0-86e5-3f747937e973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:12:49.708690 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 654/1313] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:12:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:12:49.763887 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb1ced-98ee-4393-ae8d-5de3549374ba None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:12:49.764268 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb1ced-98ee-4393-ae8d-5de3549374ba None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:12:49.764614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb1ced-98ee-4393-ae8d-5de3549374ba None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:12:49.766477 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 660/1314] 10.222.0.81 () {54 vars in 964 bytes} [Tue Jun 7 14:12:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:12:49.772892 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1ef52-442f-4f4c-a095-04ffd32fc824 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:12:49.773197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1ef52-442f-4f4c-a095-04ffd32fc824 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:12:49.773462 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1ef52-442f-4f4c-a095-04ffd32fc824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:12:49.784882 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:12:49.859083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-fbe1ef52-442f-4f4c-a095-04ffd32fc824 None None] MFA Rules not processed for user `a6a08eaecbcd41228571739d8387882a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:12:49.860360 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fbe1ef52-442f-4f4c-a095-04ffd32fc824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:12:49.949593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 655/1315] 10.222.0.81 () {58 vars in 1061 bytes} [Tue Jun 7 14:12:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:12:49.958678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb1ced-98ee-4393-ae8d-5de3549374ba None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:12:49.960340 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-74bb1ced-98ee-4393-ae8d-5de3549374ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:12:50.074301 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb1ced-98ee-4393-ae8d-5de3549374ba None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:12:50.149429 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:12:50.152628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d33bb-040c-4445-b494-291f644c0af8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:12:50.153669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d33bb-040c-4445-b494-291f644c0af8 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:12:50.154437 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d33bb-040c-4445-b494-291f644c0af8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:12:50.154995 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d33bb-040c-4445-b494-291f644c0af8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:12:50.156954 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a71d33bb-040c-4445-b494-291f644c0af8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:12:50.277697 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71d33bb-040c-4445-b494-291f644c0af8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:12:50.279251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71d33bb-040c-4445-b494-291f644c0af8 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:12:50.299032 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 661/1316] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:12:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:12:53.124070 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded5d9e-9dd0-42aa-8fb6-0469b893b7d5 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:12:53.124679 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded5d9e-9dd0-42aa-8fb6-0469b893b7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:12:53.124679 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded5d9e-9dd0-42aa-8fb6-0469b893b7d5 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:12:53.125983 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 656/1317] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:12:53.131684 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052c023-d28d-4996-a2d1-eb83350548eb None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:12:53.132139 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052c023-d28d-4996-a2d1-eb83350548eb None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:12:53.132251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052c023-d28d-4996-a2d1-eb83350548eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:12:53.196815 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-5052c023-d28d-4996-a2d1-eb83350548eb None None] MFA Rules not processed for user `d91da95dd8674152b0b2f5f61a71da6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:12:53.197998 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5052c023-d28d-4996-a2d1-eb83350548eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:12:53.249587 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:12:53.294735 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 662/1318] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:12:53 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:12:53.335381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded5d9e-9dd0-42aa-8fb6-0469b893b7d5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:12:53.336939 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bded5d9e-9dd0-42aa-8fb6-0469b893b7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:12:53.439367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded5d9e-9dd0-42aa-8fb6-0469b893b7d5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:12:53.442054 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:12:53.512586 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87678ab-b933-4e0b-a456-3bedf13b9fb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:12:53.513228 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87678ab-b933-4e0b-a456-3bedf13b9fb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:12:53.513515 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87678ab-b933-4e0b-a456-3bedf13b9fb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:12:53.513790 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87678ab-b933-4e0b-a456-3bedf13b9fb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:12:53.515462 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c87678ab-b933-4e0b-a456-3bedf13b9fb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:12:53.622062 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87678ab-b933-4e0b-a456-3bedf13b9fb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:12:53.623524 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87678ab-b933-4e0b-a456-3bedf13b9fb8 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:12:53.641034 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 657/1319] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:12:53 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:12:56.503832 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e9857-db1d-42b6-a8db-f02bd5072fb1 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:12:56.504361 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e9857-db1d-42b6-a8db-f02bd5072fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:12:56.504361 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e9857-db1d-42b6-a8db-f02bd5072fb1 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:12:56.505792 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 663/1320] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:12:56.511621 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec80839-d068-46cd-9de1-91b9f3596710 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:12:56.512050 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec80839-d068-46cd-9de1-91b9f3596710 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:12:56.512188 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec80839-d068-46cd-9de1-91b9f3596710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:12:56.551234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:12:56.576253 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0ec80839-d068-46cd-9de1-91b9f3596710 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:12:56.577457 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0ec80839-d068-46cd-9de1-91b9f3596710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:12:56.671491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 658/1321] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:12:56 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:12:56.712528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e9857-db1d-42b6-a8db-f02bd5072fb1 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:12:56.714644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-cc6e9857-db1d-42b6-a8db-f02bd5072fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:12:56.815455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e9857-db1d-42b6-a8db-f02bd5072fb1 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:12:56.887393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4469c70-d54f-4668-b3c5-0e10cae82393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:12:56.888037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4469c70-d54f-4668-b3c5-0e10cae82393 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:12:56.888309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4469c70-d54f-4668-b3c5-0e10cae82393 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:12:56.888583 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4469c70-d54f-4668-b3c5-0e10cae82393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:12:56.890217 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e4469c70-d54f-4668-b3c5-0e10cae82393 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:12:56.897003 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:12:57.004676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4469c70-d54f-4668-b3c5-0e10cae82393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:12:57.006110 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4469c70-d54f-4668-b3c5-0e10cae82393 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:12:57.025724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 664/1322] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:12:56 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:12:59.962263 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e169cc-fc97-45ef-9a3a-ee777665e51c None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:12:59.962837 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e169cc-fc97-45ef-9a3a-ee777665e51c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:12:59.962837 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e169cc-fc97-45ef-9a3a-ee777665e51c None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:12:59.965184 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 659/1323] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:12:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:12:59.970672 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916f60e-633f-4b28-9dfe-d733f4d496f4 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:12:59.971083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916f60e-633f-4b28-9dfe-d733f4d496f4 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:12:59.971383 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916f60e-633f-4b28-9dfe-d733f4d496f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:00.038800 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-3916f60e-633f-4b28-9dfe-d733f4d496f4 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:13:00.040058 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3916f60e-633f-4b28-9dfe-d733f4d496f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:00.100081 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:13:00.142035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 665/1324] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:12:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:13:00.178780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e169cc-fc97-45ef-9a3a-ee777665e51c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:13:00.180372 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f6e169cc-fc97-45ef-9a3a-ee777665e51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:00.284349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e169cc-fc97-45ef-9a3a-ee777665e51c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:13:00.297524 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:13:00.360072 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ccf29-f3ec-4d0a-b7be-48666ab380df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:13:00.360719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ccf29-f3ec-4d0a-b7be-48666ab380df service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:00.360998 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ccf29-f3ec-4d0a-b7be-48666ab380df service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:00.361268 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ccf29-f3ec-4d0a-b7be-48666ab380df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:00.362907 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-184ccf29-f3ec-4d0a-b7be-48666ab380df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:00.474140 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ccf29-f3ec-4d0a-b7be-48666ab380df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:13:00.475632 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ccf29-f3ec-4d0a-b7be-48666ab380df service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:13:00.494452 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 660/1325] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:13:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:13:02.843056 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b6048-44c1-4ace-bd81-2fe8cec1189d None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:02.843650 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b6048-44c1-4ace-bd81-2fe8cec1189d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:02.843650 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b6048-44c1-4ace-bd81-2fe8cec1189d None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:02.845010 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 666/1326] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:13:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:13:02.850228 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425f290-9b79-4209-93b4-763c4369caf9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:02.850627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425f290-9b79-4209-93b4-763c4369caf9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:02.850792 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425f290-9b79-4209-93b4-763c4369caf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:02.898941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:13:03.047721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1425f290-9b79-4209-93b4-763c4369caf9 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:13:03.048941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1425f290-9b79-4209-93b4-763c4369caf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:03.149764 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 661/1327] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:13:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:13:03.186653 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b6048-44c1-4ace-bd81-2fe8cec1189d None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:13:03.188211 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-610b6048-44c1-4ace-bd81-2fe8cec1189d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:03.292719 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b6048-44c1-4ace-bd81-2fe8cec1189d None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:13:03.366727 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade92fee-b7f3-42c6-a8a4-426a5ec34bb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:13:03.367414 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade92fee-b7f3-42c6-a8a4-426a5ec34bb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:03.367697 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade92fee-b7f3-42c6-a8a4-426a5ec34bb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:03.367973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade92fee-b7f3-42c6-a8a4-426a5ec34bb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:03.369844 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ade92fee-b7f3-42c6-a8a4-426a5ec34bb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:03.387496 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:13:03.481602 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade92fee-b7f3-42c6-a8a4-426a5ec34bb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:13:03.483172 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade92fee-b7f3-42c6-a8a4-426a5ec34bb2 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:13:03.502628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 667/1328] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:13:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:13:05.837672 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd81fe-695d-4661-ab61-37b77049bb65 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:05.838211 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd81fe-695d-4661-ab61-37b77049bb65 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:05.838211 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd81fe-695d-4661-ab61-37b77049bb65 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:05.840135 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 662/1329] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:13:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:13:05.845988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee411fc-87f6-4eca-b7ec-16c992ccf3da None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:05.846411 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee411fc-87f6-4eca-b7ec-16c992ccf3da None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:05.846620 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee411fc-87f6-4eca-b7ec-16c992ccf3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:05.912948 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-2ee411fc-87f6-4eca-b7ec-16c992ccf3da None None] MFA Rules not processed for user `d91da95dd8674152b0b2f5f61a71da6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:13:05.914155 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2ee411fc-87f6-4eca-b7ec-16c992ccf3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:05.981358 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:13:06.015199 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 668/1330] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:13:05 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:13:06.056664 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd81fe-695d-4661-ab61-37b77049bb65 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:13:06.058341 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ebbd81fe-695d-4661-ab61-37b77049bb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:06.162459 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd81fe-695d-4661-ab61-37b77049bb65 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:13:06.183124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:13:06.238188 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da936b-2a7d-4550-9d1e-bf321825e313 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:13:06.239090 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da936b-2a7d-4550-9d1e-bf321825e313 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:06.239409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da936b-2a7d-4550-9d1e-bf321825e313 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:06.239685 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da936b-2a7d-4550-9d1e-bf321825e313 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:06.241321 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-93da936b-2a7d-4550-9d1e-bf321825e313 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:06.349708 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93da936b-2a7d-4550-9d1e-bf321825e313 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:13:06.351146 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93da936b-2a7d-4550-9d1e-bf321825e313 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:13:06.369897 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 663/1331] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:13:06 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:13:08.864636 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924287bb-f895-4a80-bd10-e072528dd9b8 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:08.865163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924287bb-f895-4a80-bd10-e072528dd9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:08.865163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924287bb-f895-4a80-bd10-e072528dd9b8 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:08.866551 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 669/1332] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:13:08.878254 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1d7af-7131-4fc2-846e-ae0ff9c84e10 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:08.878254 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1d7af-7131-4fc2-846e-ae0ff9c84e10 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:08.878254 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1d7af-7131-4fc2-846e-ae0ff9c84e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:08.945275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:13:08.948552 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-41a1d7af-7131-4fc2-846e-ae0ff9c84e10 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:13:08.950381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-41a1d7af-7131-4fc2-846e-ae0ff9c84e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:09.046036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 664/1333] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:13:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:13:09.082199 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924287bb-f895-4a80-bd10-e072528dd9b8 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:13:09.083760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-924287bb-f895-4a80-bd10-e072528dd9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:09.191782 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924287bb-f895-4a80-bd10-e072528dd9b8 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:13:09.272439 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b9b43-c38d-446b-9405-f5edbd3f8139 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:13:09.273093 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b9b43-c38d-446b-9405-f5edbd3f8139 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:09.273377 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b9b43-c38d-446b-9405-f5edbd3f8139 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:09.273654 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b9b43-c38d-446b-9405-f5edbd3f8139 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:09.275327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-448b9b43-c38d-446b-9405-f5edbd3f8139 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:09.303100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:13:09.385414 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b9b43-c38d-446b-9405-f5edbd3f8139 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:13:09.386848 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b9b43-c38d-446b-9405-f5edbd3f8139 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:13:09.405560 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 670/1334] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:13:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:13:11.743646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c13011-a795-4ae3-b632-75cdb9fc95ed None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:11.744213 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c13011-a795-4ae3-b632-75cdb9fc95ed None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:11.744213 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c13011-a795-4ae3-b632-75cdb9fc95ed None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:11.746155 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 665/1335] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:13:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:13:11.751433 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52cc71-f7c7-43a5-8e66-f656f3e7c670 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:11.751808 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52cc71-f7c7-43a5-8e66-f656f3e7c670 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:11.751983 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52cc71-f7c7-43a5-8e66-f656f3e7c670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:11.816623 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-2e52cc71-f7c7-43a5-8e66-f656f3e7c670 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:13:11.817838 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2e52cc71-f7c7-43a5-8e66-f656f3e7c670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:11.891550 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:13:11.914201 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 671/1336] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:13:11.953796 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c13011-a795-4ae3-b632-75cdb9fc95ed None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:13:11.955314 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e0c13011-a795-4ae3-b632-75cdb9fc95ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:12.058064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c13011-a795-4ae3-b632-75cdb9fc95ed None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:13:12.085384 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:13:12.130263 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d8a43-0836-4fa7-bf8d-e29e1f61500a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:13:12.130898 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d8a43-0836-4fa7-bf8d-e29e1f61500a service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:12.131186 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d8a43-0836-4fa7-bf8d-e29e1f61500a service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:12.131475 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d8a43-0836-4fa7-bf8d-e29e1f61500a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:12.133093 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-391d8a43-0836-4fa7-bf8d-e29e1f61500a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:12.247033 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d8a43-0836-4fa7-bf8d-e29e1f61500a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:13:12.248485 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d8a43-0836-4fa7-bf8d-e29e1f61500a service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:13:12.269306 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 666/1337] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:13:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:13:14.905860 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab36db-1feb-4682-94d1-8fa9113c1cf9 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:14.906425 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab36db-1feb-4682-94d1-8fa9113c1cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:14.906425 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab36db-1feb-4682-94d1-8fa9113c1cf9 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:14.907826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 672/1338] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:13:14.913054 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8235cb1-1ae2-480e-8c84-72b66578573d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:14.913583 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8235cb1-1ae2-480e-8c84-72b66578573d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:14.913868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8235cb1-1ae2-480e-8c84-72b66578573d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:14.981070 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b8235cb1-1ae2-480e-8c84-72b66578573d None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:13:14.982257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b8235cb1-1ae2-480e-8c84-72b66578573d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:14.990649 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:13:15.078474 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 667/1339] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:13:14 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:13:15.115279 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab36db-1feb-4682-94d1-8fa9113c1cf9 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:13:15.116797 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1dab36db-1feb-4682-94d1-8fa9113c1cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:15.220001 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab36db-1feb-4682-94d1-8fa9113c1cf9 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:13:15.298448 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131c2ca-a3c7-4339-a774-60105ad2941f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:13:15.299096 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131c2ca-a3c7-4339-a774-60105ad2941f service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:15.299414 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131c2ca-a3c7-4339-a774-60105ad2941f service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:15.299674 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131c2ca-a3c7-4339-a774-60105ad2941f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:15.301404 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d131c2ca-a3c7-4339-a774-60105ad2941f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:15.349267 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:13:15.425324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131c2ca-a3c7-4339-a774-60105ad2941f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:13:15.426680 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131c2ca-a3c7-4339-a774-60105ad2941f service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:13:15.445637 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 673/1340] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:13:15 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:13:20.043149 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f62248-934d-4c2b-9a9e-0e7faebede3a None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:20.043717 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f62248-934d-4c2b-9a9e-0e7faebede3a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:20.043717 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f62248-934d-4c2b-9a9e-0e7faebede3a None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:20.045033 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 668/1341] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:13:20.050450 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe84cb-55fa-4014-956e-b20b066357f3 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:20.050881 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe84cb-55fa-4014-956e-b20b066357f3 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:20.050984 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe84cb-55fa-4014-956e-b20b066357f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:20.121134 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-7bbe84cb-55fa-4014-956e-b20b066357f3 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:13:20.122307 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7bbe84cb-55fa-4014-956e-b20b066357f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:20.216093 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:13:20.230637 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 674/1342] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:13:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:13:20.271029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f62248-934d-4c2b-9a9e-0e7faebede3a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:13:20.273281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d0f62248-934d-4c2b-9a9e-0e7faebede3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:20.370961 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f62248-934d-4c2b-9a9e-0e7faebede3a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:13:20.374608 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97a64f-44f4-477b-a37f-db53a4200421 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:13:20.375278 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97a64f-44f4-477b-a37f-db53a4200421 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:20.375559 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97a64f-44f4-477b-a37f-db53a4200421 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:20.375829 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97a64f-44f4-477b-a37f-db53a4200421 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:20.385917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97a64f-44f4-477b-a37f-db53a4200421 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:13:20.387631 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97a64f-44f4-477b-a37f-db53a4200421 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:13:20.397548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 669/1343] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:13:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:13:23.296215 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf270f0-5b28-4951-91c1-5e1802082c57 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:23.296728 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf270f0-5b28-4951-91c1-5e1802082c57 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:23.296728 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf270f0-5b28-4951-91c1-5e1802082c57 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:23.298089 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 675/1344] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:13:23.303574 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed47678-8093-4f6d-8bb9-a9e346920677 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:23.303964 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed47678-8093-4f6d-8bb9-a9e346920677 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:23.305440 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed47678-8093-4f6d-8bb9-a9e346920677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:23.338637 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7ed47678-8093-4f6d-8bb9-a9e346920677 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:13:23.339959 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7ed47678-8093-4f6d-8bb9-a9e346920677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:23.368425 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:13:23.421913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 670/1345] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:13:23 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:13:23.459544 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf270f0-5b28-4951-91c1-5e1802082c57 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:13:23.470495 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf270f0-5b28-4951-91c1-5e1802082c57 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:13:23.474090 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f21ea9-a56d-4c4c-81c9-66a17999b7c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:13:23.474804 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f21ea9-a56d-4c4c-81c9-66a17999b7c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:23.475160 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f21ea9-a56d-4c4c-81c9-66a17999b7c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:23.475533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f21ea9-a56d-4c4c-81c9-66a17999b7c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:23.485997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f21ea9-a56d-4c4c-81c9-66a17999b7c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:13:23.487567 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f21ea9-a56d-4c4c-81c9-66a17999b7c2 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:13:23.498116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 676/1346] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:13:23 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:13:27.725185 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03841e-9e6d-4706-8fb2-f9f82a904d8b None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:27.725618 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03841e-9e6d-4706-8fb2-f9f82a904d8b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:27.725805 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03841e-9e6d-4706-8fb2-f9f82a904d8b None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:27.727632 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 671/1347] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:13:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:13:27.733795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e595b-24df-4c4d-81c7-e3b1a14c169c None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:27.734238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e595b-24df-4c4d-81c7-e3b1a14c169c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:27.734498 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e595b-24df-4c4d-81c7-e3b1a14c169c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:27.756628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-de7e595b-24df-4c4d-81c7-e3b1a14c169c None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:13:27.757835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-de7e595b-24df-4c4d-81c7-e3b1a14c169c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:27.766623 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 677/1348] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:13:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:13:27.806994 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03841e-9e6d-4706-8fb2-f9f82a904d8b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:13:27.818396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03841e-9e6d-4706-8fb2-f9f82a904d8b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:13:27.821897 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b879e6ce-9233-4030-a1ae-0a070756d22c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:13:27.822921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879e6ce-9233-4030-a1ae-0a070756d22c service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:27.823486 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879e6ce-9233-4030-a1ae-0a070756d22c service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:27.823770 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879e6ce-9233-4030-a1ae-0a070756d22c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:27.833985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879e6ce-9233-4030-a1ae-0a070756d22c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:13:27.835435 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879e6ce-9233-4030-a1ae-0a070756d22c service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:13:27.846213 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 672/1349] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:13:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:13:33.399555 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7321467-111b-40a1-ad9f-7b21481965a9 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:33.400476 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7321467-111b-40a1-ad9f-7b21481965a9 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:33.400922 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7321467-111b-40a1-ad9f-7b21481965a9 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:33.402743 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 678/1350] 10.222.0.81 () {54 vars in 921 bytes} [Tue Jun 7 14:13:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:13:33.408125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7dddde-8a7f-4f88-9285-63a9470ab3d2 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:33.409232 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7dddde-8a7f-4f88-9285-63a9470ab3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:33.409899 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7dddde-8a7f-4f88-9285-63a9470ab3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:33.546288 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5d7dddde-8a7f-4f88-9285-63a9470ab3d2 None None] MFA Rules not processed for user `ab73cf3febbf4c768cd0116306f16c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:13:33.547596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5d7dddde-8a7f-4f88-9285-63a9470ab3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:33.634996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 673/1351] 10.222.0.81 () {58 vars in 1018 bytes} [Tue Jun 7 14:13:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:13:37.750492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=13 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:13:38.062047 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73361dbe-8a97-41e1-90c7-7f8d0e8b461b None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:38.062240 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73361dbe-8a97-41e1-90c7-7f8d0e8b461b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:38.062532 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73361dbe-8a97-41e1-90c7-7f8d0e8b461b None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:38.064215 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 679/1352] 10.222.0.81 () {54 vars in 921 bytes} [Tue Jun 7 14:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:13:38.069410 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3359ea-7b12-460c-a9a6-8f537876b075 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:38.069872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3359ea-7b12-460c-a9a6-8f537876b075 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:38.070143 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3359ea-7b12-460c-a9a6-8f537876b075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:38.089773 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2f3359ea-7b12-460c-a9a6-8f537876b075 None None] MFA Rules not processed for user `ab73cf3febbf4c768cd0116306f16c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:13:38.090977 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2f3359ea-7b12-460c-a9a6-8f537876b075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:38.099896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 674/1353] 10.222.0.81 () {58 vars in 1018 bytes} [Tue Jun 7 14:13:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:13:39.798080 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06b256-244a-4a15-862a-ceb16a24d7bc None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:39.798707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06b256-244a-4a15-862a-ceb16a24d7bc None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:39.798707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06b256-244a-4a15-862a-ceb16a24d7bc None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:39.800035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 680/1354] 10.222.0.81 () {54 vars in 921 bytes} [Tue Jun 7 14:13:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:13:39.804987 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8cf3f-a343-482e-a399-0e60b182fb01 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:39.805399 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8cf3f-a343-482e-a399-0e60b182fb01 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:39.805671 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8cf3f-a343-482e-a399-0e60b182fb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:39.825370 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-27c8cf3f-a343-482e-a399-0e60b182fb01 None None] MFA Rules not processed for user `ab73cf3febbf4c768cd0116306f16c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:13:39.826606 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-27c8cf3f-a343-482e-a399-0e60b182fb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:39.835689 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 675/1355] 10.222.0.81 () {58 vars in 1018 bytes} [Tue Jun 7 14:13:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:13:43.700634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895431a-3195-4deb-867e-be0194e0eb35 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:43.701196 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895431a-3195-4deb-867e-be0194e0eb35 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:43.701196 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895431a-3195-4deb-867e-be0194e0eb35 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:43.702476 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 681/1356] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:13:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:13:43.707796 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e140d82-a63d-4f6b-b4e1-e2a36e315a78 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:43.708213 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e140d82-a63d-4f6b-b4e1-e2a36e315a78 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:43.708341 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e140d82-a63d-4f6b-b4e1-e2a36e315a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:43.726491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5e140d82-a63d-4f6b-b4e1-e2a36e315a78 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:13:43.728289 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5e140d82-a63d-4f6b-b4e1-e2a36e315a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:43.736739 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 676/1357] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:13:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:13:43.791415 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862b930-f680-407d-9f34-839488373dce None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:43.792182 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862b930-f680-407d-9f34-839488373dce None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:43.792674 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862b930-f680-407d-9f34-839488373dce None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:43.794675 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 682/1358] 10.222.0.81 () {54 vars in 961 bytes} [Tue Jun 7 14:13:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:13:43.814127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94089d-c812-4221-9d30-cdd1063a617a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:43.814765 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94089d-c812-4221-9d30-cdd1063a617a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:43.815401 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94089d-c812-4221-9d30-cdd1063a617a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:43.859389 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:13:43.869751 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8b94089d-c812-4221-9d30-cdd1063a617a None None] MFA Rules not processed for user `2ae726fbc9cf40e68c7587552a118a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:13:43.870952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8b94089d-c812-4221-9d30-cdd1063a617a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:43.959052 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 677/1359] 10.222.0.81 () {58 vars in 1058 bytes} [Tue Jun 7 14:13:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:13:43.967432 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862b930-f680-407d-9f34-839488373dce None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:13:43.983493 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862b930-f680-407d-9f34-839488373dce None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:13:43.989688 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bce964-b11d-43e3-8e91-e7e11294cd57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:13:43.990343 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bce964-b11d-43e3-8e91-e7e11294cd57 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:43.990630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bce964-b11d-43e3-8e91-e7e11294cd57 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:43.990909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bce964-b11d-43e3-8e91-e7e11294cd57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:43.999900 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bce964-b11d-43e3-8e91-e7e11294cd57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:13:44.001487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bce964-b11d-43e3-8e91-e7e11294cd57 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:13:44.012446 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 683/1360] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:13:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:13:46.518783 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b3f20-757f-4ef6-9e6a-40f014b3be2c None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:46.519359 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b3f20-757f-4ef6-9e6a-40f014b3be2c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:46.519359 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b3f20-757f-4ef6-9e6a-40f014b3be2c None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:46.520886 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 678/1361] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:13:46.528152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf89718-e978-42a5-a150-d2633bf21606 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:46.528627 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf89718-e978-42a5-a150-d2633bf21606 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:46.528829 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf89718-e978-42a5-a150-d2633bf21606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:46.549045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-adf89718-e978-42a5-a150-d2633bf21606 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:13:46.550376 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-adf89718-e978-42a5-a150-d2633bf21606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:46.558881 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 684/1362] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:13:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:13:46.603989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b3f20-757f-4ef6-9e6a-40f014b3be2c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:13:46.618510 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b3f20-757f-4ef6-9e6a-40f014b3be2c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:13:46.624021 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d28f9c-33f8-4346-81b2-0d7ac85234ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:13:46.624656 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d28f9c-33f8-4346-81b2-0d7ac85234ad service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:46.624934 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d28f9c-33f8-4346-81b2-0d7ac85234ad service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:46.625204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d28f9c-33f8-4346-81b2-0d7ac85234ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:46.635019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d28f9c-33f8-4346-81b2-0d7ac85234ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:13:46.636624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d28f9c-33f8-4346-81b2-0d7ac85234ad service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:13:46.646536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 679/1363] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:13:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:13:49.254305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2df5f1-871e-4ee0-8841-46cc52bb6926 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:49.254725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2df5f1-871e-4ee0-8841-46cc52bb6926 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:49.254858 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2df5f1-871e-4ee0-8841-46cc52bb6926 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:49.256281 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 685/1364] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:13:49.262320 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f45cd-6aaa-4d0c-b28c-845001144af4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:49.262736 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f45cd-6aaa-4d0c-b28c-845001144af4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:49.262852 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f45cd-6aaa-4d0c-b28c-845001144af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:49.280356 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-663f45cd-6aaa-4d0c-b28c-845001144af4 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:13:49.281549 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-663f45cd-6aaa-4d0c-b28c-845001144af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:49.298450 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 680/1365] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:13:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:13:49.335702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ccc6b-c18f-4813-8a25-bd1865959bb4 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:49.336023 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ccc6b-c18f-4813-8a25-bd1865959bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:49.336298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ccc6b-c18f-4813-8a25-bd1865959bb4 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:49.338701 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 686/1366] 10.222.0.81 () {54 vars in 961 bytes} [Tue Jun 7 14:13:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:13:49.354239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72eada-2ad1-4dc8-9249-398e8e3af783 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:49.354859 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72eada-2ad1-4dc8-9249-398e8e3af783 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:49.355394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72eada-2ad1-4dc8-9249-398e8e3af783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:49.375318 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-cb72eada-2ad1-4dc8-9249-398e8e3af783 None None] MFA Rules not processed for user `2ae726fbc9cf40e68c7587552a118a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:13:49.376897 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cb72eada-2ad1-4dc8-9249-398e8e3af783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:49.386611 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 681/1367] 10.222.0.81 () {58 vars in 1058 bytes} [Tue Jun 7 14:13:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:13:49.395010 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ccc6b-c18f-4813-8a25-bd1865959bb4 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:13:49.410769 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ccc6b-c18f-4813-8a25-bd1865959bb4 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:13:49.416880 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82079ee8-920d-470f-88aa-76b4fa2d133a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:13:49.417570 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82079ee8-920d-470f-88aa-76b4fa2d133a service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:49.417808 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82079ee8-920d-470f-88aa-76b4fa2d133a service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:49.418082 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82079ee8-920d-470f-88aa-76b4fa2d133a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:49.426301 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82079ee8-920d-470f-88aa-76b4fa2d133a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:13:49.427922 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82079ee8-920d-470f-88aa-76b4fa2d133a service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:13:49.437609 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 687/1368] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:13:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:13:51.775502 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8344c-64f5-409a-aa12-1ff6d47585b3 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:51.776443 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8344c-64f5-409a-aa12-1ff6d47585b3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:51.776753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8344c-64f5-409a-aa12-1ff6d47585b3 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:51.778472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 682/1369] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:13:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:13:51.784358 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc5483-38ea-4bba-9777-ec443db7ed0f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:51.784892 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc5483-38ea-4bba-9777-ec443db7ed0f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:51.784892 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc5483-38ea-4bba-9777-ec443db7ed0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:51.803355 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-25cc5483-38ea-4bba-9777-ec443db7ed0f None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:13:51.804552 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-25cc5483-38ea-4bba-9777-ec443db7ed0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:51.813826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 688/1370] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:13:51.858771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8344c-64f5-409a-aa12-1ff6d47585b3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:13:51.874074 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8344c-64f5-409a-aa12-1ff6d47585b3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:13:51.879686 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c7a1b-7f68-41fa-b69f-9068bf04d9b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:13:51.880435 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c7a1b-7f68-41fa-b69f-9068bf04d9b9 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:51.880720 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c7a1b-7f68-41fa-b69f-9068bf04d9b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:51.880993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c7a1b-7f68-41fa-b69f-9068bf04d9b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:51.890428 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1c7a1b-7f68-41fa-b69f-9068bf04d9b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:13:51.892035 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1c7a1b-7f68-41fa-b69f-9068bf04d9b9 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:13:51.902700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 683/1371] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:13:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:13:52.099317 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650d2c8-79e4-4cf4-8621-2cf6fa33489c None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:52.099914 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650d2c8-79e4-4cf4-8621-2cf6fa33489c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:52.100196 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650d2c8-79e4-4cf4-8621-2cf6fa33489c None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:52.101626 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 689/1372] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:13:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:13:52.107869 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7ae0f-a54f-4f54-8522-50f6ece8949f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:52.108229 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7ae0f-a54f-4f54-8522-50f6ece8949f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:52.108427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7ae0f-a54f-4f54-8522-50f6ece8949f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:52.127403 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6fe7ae0f-a54f-4f54-8522-50f6ece8949f None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:13:52.128645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6fe7ae0f-a54f-4f54-8522-50f6ece8949f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:52.137590 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 684/1373] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:13:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:13:52.444032 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a6830-8b6f-4b15-9f98-bc184b754c58 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:52.444330 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a6830-8b6f-4b15-9f98-bc184b754c58 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:52.444613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a6830-8b6f-4b15-9f98-bc184b754c58 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:52.446198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 690/1374] 10.222.0.81 () {54 vars in 963 bytes} [Tue Jun 7 14:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:13:52.451036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceaf8da-fe23-4062-afba-27cd3f31243c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:52.451365 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceaf8da-fe23-4062-afba-27cd3f31243c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:52.452077 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceaf8da-fe23-4062-afba-27cd3f31243c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:52.504582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5ceaf8da-fe23-4062-afba-27cd3f31243c None None] MFA Rules not processed for user `38b1ff21b8e5445b878740a327e4442f`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:13:52.505975 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5ceaf8da-fe23-4062-afba-27cd3f31243c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:52.570775 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 685/1375] 10.222.0.81 () {58 vars in 1060 bytes} [Tue Jun 7 14:13:52 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:13:52.578429 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a6830-8b6f-4b15-9f98-bc184b754c58 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:13:52.591658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a6830-8b6f-4b15-9f98-bc184b754c58 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:13:52.595743 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078a188-b136-4284-accd-72915e38a348 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:13:52.596387 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078a188-b136-4284-accd-72915e38a348 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:52.596665 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078a188-b136-4284-accd-72915e38a348 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:52.596938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078a188-b136-4284-accd-72915e38a348 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:52.607619 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b078a188-b136-4284-accd-72915e38a348 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:13:52.609043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b078a188-b136-4284-accd-72915e38a348 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:13:52.620504 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 691/1376] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:13:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:13:54.276147 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783630f-498c-41cb-aff5-3f1bce8837b5 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:54.276710 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783630f-498c-41cb-aff5-3f1bce8837b5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:54.276710 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783630f-498c-41cb-aff5-3f1bce8837b5 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:54.278097 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 686/1377] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:13:54.283478 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24ae25-1f06-4895-ac8b-1858dd0d8a8a None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:54.283917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24ae25-1f06-4895-ac8b-1858dd0d8a8a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:54.284033 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24ae25-1f06-4895-ac8b-1858dd0d8a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:54.302539 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-cc24ae25-1f06-4895-ac8b-1858dd0d8a8a None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:13:54.303777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-cc24ae25-1f06-4895-ac8b-1858dd0d8a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:54.312514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 692/1378] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:13:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:13:54.360914 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783630f-498c-41cb-aff5-3f1bce8837b5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:13:54.377456 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783630f-498c-41cb-aff5-3f1bce8837b5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:13:54.383581 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257de81-dd10-4016-8754-ca2940171c69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:13:54.384356 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257de81-dd10-4016-8754-ca2940171c69 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:54.384645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257de81-dd10-4016-8754-ca2940171c69 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:54.384922 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257de81-dd10-4016-8754-ca2940171c69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:54.394258 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257de81-dd10-4016-8754-ca2940171c69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:13:54.395927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257de81-dd10-4016-8754-ca2940171c69 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:13:54.400436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:13:54.407989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 687/1379] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:13:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:13:56.753176 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b04ce-d770-4f86-aec9-25ade5f49c97 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:56.753757 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b04ce-d770-4f86-aec9-25ade5f49c97 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:56.753757 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b04ce-d770-4f86-aec9-25ade5f49c97 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:56.755105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 693/1380] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:13:56.761133 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0959baa-ba23-4256-ba3d-1489e0390357 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:56.761564 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0959baa-ba23-4256-ba3d-1489e0390357 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:56.761687 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0959baa-ba23-4256-ba3d-1489e0390357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:56.780342 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b0959baa-ba23-4256-ba3d-1489e0390357 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:13:56.781529 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b0959baa-ba23-4256-ba3d-1489e0390357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:56.789634 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 688/1381] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:13:56.834711 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b04ce-d770-4f86-aec9-25ade5f49c97 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:13:56.850564 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b04ce-d770-4f86-aec9-25ade5f49c97 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:13:56.856205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0458828-72eb-4258-ac8f-b7a07aa51c7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:13:56.856852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0458828-72eb-4258-ac8f-b7a07aa51c7a service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:56.857142 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0458828-72eb-4258-ac8f-b7a07aa51c7a service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:56.857424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0458828-72eb-4258-ac8f-b7a07aa51c7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:56.866573 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0458828-72eb-4258-ac8f-b7a07aa51c7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:13:56.868856 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0458828-72eb-4258-ac8f-b7a07aa51c7a service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:13:56.878948 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 694/1382] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:13:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:13:59.200262 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047738ae-6555-4523-85b3-de765dadfe99 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:59.200784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047738ae-6555-4523-85b3-de765dadfe99 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:59.200784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047738ae-6555-4523-85b3-de765dadfe99 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:59.202077 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 689/1383] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:13:59.209300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7da360-6faf-49cc-aad6-d9d4d4431841 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:59.209840 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7da360-6faf-49cc-aad6-d9d4d4431841 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:59.210137 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7da360-6faf-49cc-aad6-d9d4d4431841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:59.230917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-2d7da360-6faf-49cc-aad6-d9d4d4431841 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:13:59.232326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2d7da360-6faf-49cc-aad6-d9d4d4431841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:13:59.247739 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 695/1384] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:13:59.287003 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047738ae-6555-4523-85b3-de765dadfe99 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:13:59.302016 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047738ae-6555-4523-85b3-de765dadfe99 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:13:59.308369 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154812e6-3f6b-4ade-8198-daf883e751d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:13:59.309028 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154812e6-3f6b-4ade-8198-daf883e751d9 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:13:59.309307 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154812e6-3f6b-4ade-8198-daf883e751d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:13:59.309579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154812e6-3f6b-4ade-8198-daf883e751d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:13:59.318555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154812e6-3f6b-4ade-8198-daf883e751d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:13:59.320243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154812e6-3f6b-4ade-8198-daf883e751d9 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:13:59.331011 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 690/1385] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:13:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:14:01.634147 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3fd3d-78ee-4830-a7ad-5e9a9ad96380 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:01.634734 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3fd3d-78ee-4830-a7ad-5e9a9ad96380 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:01.634734 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3fd3d-78ee-4830-a7ad-5e9a9ad96380 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:01.636373 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 696/1386] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:14:01.641811 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90ac19-c080-47bd-ac1c-7b7683704e47 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:01.642516 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90ac19-c080-47bd-ac1c-7b7683704e47 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:01.642516 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90ac19-c080-47bd-ac1c-7b7683704e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:01.660835 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3a90ac19-c080-47bd-ac1c-7b7683704e47 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:14:01.662021 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3a90ac19-c080-47bd-ac1c-7b7683704e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:14:01.670537 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 691/1387] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:14:01.719890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3fd3d-78ee-4830-a7ad-5e9a9ad96380 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:14:01.737901 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3fd3d-78ee-4830-a7ad-5e9a9ad96380 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:14:01.744369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b24d4-d87f-4005-9855-7d1cec41ad38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:14:01.745159 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b24d4-d87f-4005-9855-7d1cec41ad38 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:01.745569 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b24d4-d87f-4005-9855-7d1cec41ad38 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:01.745911 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b24d4-d87f-4005-9855-7d1cec41ad38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:01.756902 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b24d4-d87f-4005-9855-7d1cec41ad38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:14:01.759546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b24d4-d87f-4005-9855-7d1cec41ad38 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:14:01.771847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 697/1388] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:14:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:14:04.088154 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfed20-5941-4868-b965-b1eea4be294d None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:04.088749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfed20-5941-4868-b965-b1eea4be294d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:04.088749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfed20-5941-4868-b965-b1eea4be294d None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:04.090166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 692/1389] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:14:04.096820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce7234-2011-40db-8c61-b34a869e15b7 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:04.097621 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce7234-2011-40db-8c61-b34a869e15b7 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:04.097900 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce7234-2011-40db-8c61-b34a869e15b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:04.116708 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-5fce7234-2011-40db-8c61-b34a869e15b7 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:14:04.118000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5fce7234-2011-40db-8c61-b34a869e15b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:14:04.126261 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 698/1390] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:14:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:14:04.178467 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bfed20-5941-4868-b965-b1eea4be294d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:14:04.197203 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bfed20-5941-4868-b965-b1eea4be294d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:14:04.202666 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3258992d-153d-4774-b9d8-0af99481996b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:14:04.203385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258992d-153d-4774-b9d8-0af99481996b service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:04.203684 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258992d-153d-4774-b9d8-0af99481996b service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:04.203971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258992d-153d-4774-b9d8-0af99481996b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:04.217453 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3258992d-153d-4774-b9d8-0af99481996b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:14:04.219539 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3258992d-153d-4774-b9d8-0af99481996b service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:14:04.233766 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 693/1391] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:14:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:14:06.815797 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75029c0-dcd0-4dd4-b94e-58e1527e764f None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:06.816409 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75029c0-dcd0-4dd4-b94e-58e1527e764f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:06.816409 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75029c0-dcd0-4dd4-b94e-58e1527e764f None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:06.817732 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 699/1392] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:14:06.823209 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae9dac-a8b8-44cb-a840-2d3c9c83a2ea None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:06.823675 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae9dac-a8b8-44cb-a840-2d3c9c83a2ea None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:06.823817 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae9dac-a8b8-44cb-a840-2d3c9c83a2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:06.842043 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e2ae9dac-a8b8-44cb-a840-2d3c9c83a2ea None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:14:06.843259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e2ae9dac-a8b8-44cb-a840-2d3c9c83a2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:14:06.851075 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 694/1393] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:14:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:14:06.896367 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75029c0-dcd0-4dd4-b94e-58e1527e764f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:14:06.911539 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75029c0-dcd0-4dd4-b94e-58e1527e764f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:14:06.917129 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd52b68-f584-4f4c-ab48-fa3251d8f681 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:14:06.917769 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd52b68-f584-4f4c-ab48-fa3251d8f681 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:06.918053 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd52b68-f584-4f4c-ab48-fa3251d8f681 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:06.918445 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd52b68-f584-4f4c-ab48-fa3251d8f681 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:06.927899 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd52b68-f584-4f4c-ab48-fa3251d8f681 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:14:06.929474 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd52b68-f584-4f4c-ab48-fa3251d8f681 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:14:06.939780 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 700/1394] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:14:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:14:08.567102 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03b352-974e-42a4-8962-9038e654d133 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:08.567700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03b352-974e-42a4-8962-9038e654d133 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:08.567700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03b352-974e-42a4-8962-9038e654d133 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:08.569112 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 695/1395] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:14:08.575067 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d10984-eda8-4826-a466-c974b5b84802 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:08.575525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d10984-eda8-4826-a466-c974b5b84802 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:08.575673 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d10984-eda8-4826-a466-c974b5b84802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:08.581817 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=52 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:14:08.594854 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-21d10984-eda8-4826-a466-c974b5b84802 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:14:08.596091 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-21d10984-eda8-4826-a466-c974b5b84802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:14:08.604100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 701/1396] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:14:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:14:08.660300 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55f30a-a097-40fd-a729-b43b6e9cc17b None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:08.660656 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55f30a-a097-40fd-a729-b43b6e9cc17b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:08.661015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55f30a-a097-40fd-a729-b43b6e9cc17b None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:08.662456 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 696/1397] 10.222.0.81 () {54 vars in 962 bytes} [Tue Jun 7 14:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:14:08.667927 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e7303-2169-415e-af87-a9d97e7c9aed None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:08.668220 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e7303-2169-415e-af87-a9d97e7c9aed None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:08.668529 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e7303-2169-415e-af87-a9d97e7c9aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:08.717260 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-650e7303-2169-415e-af87-a9d97e7c9aed None None] MFA Rules not processed for user `b6180393da0f495db964f0b2588d8b90`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:14:08.718459 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-650e7303-2169-415e-af87-a9d97e7c9aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:14:08.779400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 702/1398] 10.222.0.81 () {58 vars in 1059 bytes} [Tue Jun 7 14:14:08 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:14:08.786880 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55f30a-a097-40fd-a729-b43b6e9cc17b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:14:08.797634 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55f30a-a097-40fd-a729-b43b6e9cc17b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:14:08.801361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc7c465-da4a-4e5e-95d9-f1c692eb0557 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6180393da0f495db964f0b2588d8b90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:14:08.802023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7c465-da4a-4e5e-95d9-f1c692eb0557 service glance] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:08.802306 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7c465-da4a-4e5e-95d9-f1c692eb0557 service glance] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:08.802585 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7c465-da4a-4e5e-95d9-f1c692eb0557 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:08.812521 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc7c465-da4a-4e5e-95d9-f1c692eb0557 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:14:08.813940 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc7c465-da4a-4e5e-95d9-f1c692eb0557 service glance] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:14:08.824199 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 697/1399] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:14:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:14:09.272325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30fd2f-99a2-4181-9d24-2512fcfdfeab None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:09.272614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30fd2f-99a2-4181-9d24-2512fcfdfeab None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:09.272901 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30fd2f-99a2-4181-9d24-2512fcfdfeab None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:09.274475 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 703/1400] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:14:09.279810 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef60529-c7e6-4f86-8c3f-a92bfa1f1700 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:09.280122 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef60529-c7e6-4f86-8c3f-a92bfa1f1700 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:09.280403 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef60529-c7e6-4f86-8c3f-a92bfa1f1700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:09.299718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2ef60529-c7e6-4f86-8c3f-a92bfa1f1700 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:14:09.300925 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2ef60529-c7e6-4f86-8c3f-a92bfa1f1700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:14:09.309530 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 698/1401] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:14:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:14:09.358346 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30fd2f-99a2-4181-9d24-2512fcfdfeab None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:14:09.377368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30fd2f-99a2-4181-9d24-2512fcfdfeab None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:14:09.380413 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8c351-3e25-4c6f-b697-2fb5daab58f7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:09.380707 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8c351-3e25-4c6f-b697-2fb5daab58f7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:09.380987 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8c351-3e25-4c6f-b697-2fb5daab58f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:09.383598 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3de4ac-195e-49fb-a7c3-f0ac3d95f0bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:14:09.384418 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3de4ac-195e-49fb-a7c3-f0ac3d95f0bd service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:09.384700 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3de4ac-195e-49fb-a7c3-f0ac3d95f0bd service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:09.384978 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3de4ac-195e-49fb-a7c3-f0ac3d95f0bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:09.393451 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3de4ac-195e-49fb-a7c3-f0ac3d95f0bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:14:09.395034 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3de4ac-195e-49fb-a7c3-f0ac3d95f0bd service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:14:09.405824 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 704/1402] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:14:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:14:09.442608 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0eb8c351-3e25-4c6f-b697-2fb5daab58f7 None None] MFA Rules not processed for user `d00b8bc311234e6da47b365ac8a7183a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:14:09.443963 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0eb8c351-3e25-4c6f-b697-2fb5daab58f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:14:09.515340 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 699/1403] 10.222.0.81 () {58 vars in 1009 bytes} [Tue Jun 7 14:14:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:14:09.562505 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f875801-ddab-492f-910a-458ab4b6de02 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:09.562814 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f875801-ddab-492f-910a-458ab4b6de02 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:09.563100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f875801-ddab-492f-910a-458ab4b6de02 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:09.564683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 705/1404] 10.222.0.81 () {54 vars in 942 bytes} [Tue Jun 7 14:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:14:09.572843 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427bec6-9597-44f4-9753-6f40763cba80 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:09.573389 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427bec6-9597-44f4-9753-6f40763cba80 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:09.573389 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427bec6-9597-44f4-9753-6f40763cba80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:09.627549 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6427bec6-9597-44f4-9753-6f40763cba80 None None] MFA Rules not processed for user `da7c0f99060a44eaab743ba71f159e88`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:14:09.628684 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6427bec6-9597-44f4-9753-6f40763cba80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:14:09.665599 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:14:09.710878 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 700/1405] 10.222.0.81 () {58 vars in 1039 bytes} [Tue Jun 7 14:14:09 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:14:09.720514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f875801-ddab-492f-910a-458ab4b6de02 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:14:09.733913 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f875801-ddab-492f-910a-458ab4b6de02 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:14:09.739325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de0d33-0b1f-4c65-9a15-d696dc6c976c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da7c0f99060a44eaab743ba71f159e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:14:09.740333 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de0d33-0b1f-4c65-9a15-d696dc6c976c service swift] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:09.740657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de0d33-0b1f-4c65-9a15-d696dc6c976c service swift] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:09.740997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de0d33-0b1f-4c65-9a15-d696dc6c976c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:09.750455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de0d33-0b1f-4c65-9a15-d696dc6c976c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:14:09.752106 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de0d33-0b1f-4c65-9a15-d696dc6c976c service swift] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:14:09.761068 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 706/1406] 10.222.0.81 () {60 vars in 1384 bytes} [Tue Jun 7 14:14:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:14:11.736051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3d142-0711-48d4-bc7e-5b5d36be0565 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:11.736602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3d142-0711-48d4-bc7e-5b5d36be0565 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:11.736602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3d142-0711-48d4-bc7e-5b5d36be0565 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:11.738232 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 701/1407] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:14:11.744306 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a2d26-8c7e-493b-a620-dc641e494c91 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:11.745234 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a2d26-8c7e-493b-a620-dc641e494c91 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:11.745539 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a2d26-8c7e-493b-a620-dc641e494c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:11.765454 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-eb7a2d26-8c7e-493b-a620-dc641e494c91 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:14:11.766748 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-eb7a2d26-8c7e-493b-a620-dc641e494c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:14:11.777928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 707/1408] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:14:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:14:11.825062 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc3d142-0711-48d4-bc7e-5b5d36be0565 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:14:11.840215 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc3d142-0711-48d4-bc7e-5b5d36be0565 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:14:11.845989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9372d-bb8c-4ded-8380-115bd0289973 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:14:11.846731 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9372d-bb8c-4ded-8380-115bd0289973 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:11.847087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9372d-bb8c-4ded-8380-115bd0289973 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:11.847466 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9372d-bb8c-4ded-8380-115bd0289973 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:11.856558 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9372d-bb8c-4ded-8380-115bd0289973 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:14:11.858223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9372d-bb8c-4ded-8380-115bd0289973 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:14:11.868821 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 702/1409] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:14:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:14:21.760506 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=55 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:14:22.185146 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=21 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:14:33.538816 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220253bd-f222-4428-92eb-e5f16de94fff None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:33.539479 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220253bd-f222-4428-92eb-e5f16de94fff None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:33.539479 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220253bd-f222-4428-92eb-e5f16de94fff None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:33.540814 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 708/1410] 10.222.0.81 () {54 vars in 918 bytes} [Tue Jun 7 14:14:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:14:33.546120 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135dbdf-48e1-4fe2-991a-6553202290d6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:33.546551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135dbdf-48e1-4fe2-991a-6553202290d6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:33.546679 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135dbdf-48e1-4fe2-991a-6553202290d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:33.564485 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8135dbdf-48e1-4fe2-991a-6553202290d6 None None] MFA Rules not processed for user `ab73cf3febbf4c768cd0116306f16c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:14:33.565688 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8135dbdf-48e1-4fe2-991a-6553202290d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:14:33.574310 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 703/1411] 10.222.0.81 () {58 vars in 1015 bytes} [Tue Jun 7 14:14:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:14:33.588949 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc4d7e-c096-463e-89f4-b7c53c1f27ef None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:33.589238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc4d7e-c096-463e-89f4-b7c53c1f27ef None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:33.589516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc4d7e-c096-463e-89f4-b7c53c1f27ef None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:33.590901 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 709/1412] 10.222.0.81 () {54 vars in 918 bytes} [Tue Jun 7 14:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:14:33.595493 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4aa3c-4f01-41c8-a064-d9bdb6d4a1ee None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:33.595783 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4aa3c-4f01-41c8-a064-d9bdb6d4a1ee None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:33.596063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4aa3c-4f01-41c8-a064-d9bdb6d4a1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:33.615150 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-87e4aa3c-4f01-41c8-a064-d9bdb6d4a1ee None None] MFA Rules not processed for user `2ae726fbc9cf40e68c7587552a118a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:14:33.616380 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-87e4aa3c-4f01-41c8-a064-d9bdb6d4a1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:14:33.625476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 704/1413] 10.222.0.81 () {58 vars in 1015 bytes} [Tue Jun 7 14:14:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:14:36.394863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5e40f-0b12-4c56-a29e-4250cdc1a86a None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:36.395431 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5e40f-0b12-4c56-a29e-4250cdc1a86a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:36.395539 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5e40f-0b12-4c56-a29e-4250cdc1a86a None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:36.396919 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:14:36.402540 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd60e16-9342-4bd2-b231-1744c9b08adc None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:36.402952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd60e16-9342-4bd2-b231-1744c9b08adc None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:36.403091 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd60e16-9342-4bd2-b231-1744c9b08adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:36.421842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-afd60e16-9342-4bd2-b231-1744c9b08adc None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:14:36.422976 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-afd60e16-9342-4bd2-b231-1744c9b08adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:14:36.431309 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:14:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:14:36.474035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5e40f-0b12-4c56-a29e-4250cdc1a86a None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:14:36.489680 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5e40f-0b12-4c56-a29e-4250cdc1a86a None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:14:36.495925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03047e-01d4-43f0-9585-b835535fb552 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:14:36.496573 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03047e-01d4-43f0-9585-b835535fb552 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:36.496860 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03047e-01d4-43f0-9585-b835535fb552 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:36.497139 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03047e-01d4-43f0-9585-b835535fb552 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:36.505726 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c03047e-01d4-43f0-9585-b835535fb552 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:14:36.507546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c03047e-01d4-43f0-9585-b835535fb552 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:14:36.517026 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 711/1416] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:14:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:14:37.497168 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755a408-09b1-4a06-bd4e-f82ea318f118 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:37.497720 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755a408-09b1-4a06-bd4e-f82ea318f118 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:37.497720 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755a408-09b1-4a06-bd4e-f82ea318f118 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:37.499153 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 14:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:14:37.504040 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f9024-9a20-4023-a2af-3b118d91ddf1 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:37.504480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f9024-9a20-4023-a2af-3b118d91ddf1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:37.504588 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f9024-9a20-4023-a2af-3b118d91ddf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:37.523931 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-685f9024-9a20-4023-a2af-3b118d91ddf1 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:14:37.525130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-685f9024-9a20-4023-a2af-3b118d91ddf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:14:37.532889 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 14:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:14:37.704020 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0755a408-09b1-4a06-bd4e-f82ea318f118 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:14:37.718404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0755a408-09b1-4a06-bd4e-f82ea318f118 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:14:37.723622 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6396e5-8c20-424e-8096-09fead2f3c72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:14:37.724251 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6396e5-8c20-424e-8096-09fead2f3c72 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:37.724529 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6396e5-8c20-424e-8096-09fead2f3c72 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:37.724798 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6396e5-8c20-424e-8096-09fead2f3c72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:37.734038 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6396e5-8c20-424e-8096-09fead2f3c72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:14:37.735639 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6396e5-8c20-424e-8096-09fead2f3c72 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:14:37.745986 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 707/1419] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:14:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:14:38.677715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc02f0a-3cf4-4c9b-9a8d-7691072d38a3 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:38.678256 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc02f0a-3cf4-4c9b-9a8d-7691072d38a3 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:38.678409 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc02f0a-3cf4-4c9b-9a8d-7691072d38a3 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:38.679929 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 14:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:14:38.684757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315df5a7-147a-473d-8972-1da3ff3872cc None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:38.685226 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315df5a7-147a-473d-8972-1da3ff3872cc None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:38.685335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315df5a7-147a-473d-8972-1da3ff3872cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:38.703429 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-315df5a7-147a-473d-8972-1da3ff3872cc None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:14:38.704614 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-315df5a7-147a-473d-8972-1da3ff3872cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:14:38.713083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 14:14:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:14:38.861641 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc02f0a-3cf4-4c9b-9a8d-7691072d38a3 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:14:38.876706 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc02f0a-3cf4-4c9b-9a8d-7691072d38a3 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:14:38.882652 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f4f28-52f6-4cc1-adf9-a639e0f40639 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:14:38.883327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f4f28-52f6-4cc1-adf9-a639e0f40639 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:38.883611 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f4f28-52f6-4cc1-adf9-a639e0f40639 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:38.883887 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f4f28-52f6-4cc1-adf9-a639e0f40639 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:38.892004 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054f4f28-52f6-4cc1-adf9-a639e0f40639 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:14:38.893588 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054f4f28-52f6-4cc1-adf9-a639e0f40639 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:14:38.902896 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 714/1422] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:14:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:14:39.808076 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b31df-5f70-4e00-a092-a54d7f8e22c0 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:39.808652 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b31df-5f70-4e00-a092-a54d7f8e22c0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:39.808652 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b31df-5f70-4e00-a092-a54d7f8e22c0 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:39.809996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 14:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:14:39.814904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4bb14-67a4-4617-987f-94b4a139ce4f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:39.815353 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4bb14-67a4-4617-987f-94b4a139ce4f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:39.815467 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4bb14-67a4-4617-987f-94b4a139ce4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:39.833896 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-f2a4bb14-67a4-4617-987f-94b4a139ce4f None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:14:39.835290 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f2a4bb14-67a4-4617-987f-94b4a139ce4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:14:39.842974 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 14:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:14:39.992479 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b31df-5f70-4e00-a092-a54d7f8e22c0 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:14:40.006957 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b31df-5f70-4e00-a092-a54d7f8e22c0 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:14:40.012470 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a819b-0aa9-4e29-a03f-41a7e388873c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:14:40.013104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a819b-0aa9-4e29-a03f-41a7e388873c service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:40.013388 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a819b-0aa9-4e29-a03f-41a7e388873c service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:40.013667 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a819b-0aa9-4e29-a03f-41a7e388873c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:40.021825 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a819b-0aa9-4e29-a03f-41a7e388873c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:14:40.024028 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a819b-0aa9-4e29-a03f-41a7e388873c service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:14:40.033472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 710/1425] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:14:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:14:40.955821 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef49179-afe0-49bb-86d6-87aa62a38bab None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:40.956363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef49179-afe0-49bb-86d6-87aa62a38bab None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:40.956492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef49179-afe0-49bb-86d6-87aa62a38bab None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:40.957783 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 14:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:14:40.962995 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afa1a5-15a9-44e2-8419-f40fc55ab2e5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:40.963475 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afa1a5-15a9-44e2-8419-f40fc55ab2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:40.963589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afa1a5-15a9-44e2-8419-f40fc55ab2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:40.981583 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-16afa1a5-15a9-44e2-8419-f40fc55ab2e5 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:14:40.982764 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-16afa1a5-15a9-44e2-8419-f40fc55ab2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:14:40.990559 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 14:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:14:41.144613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef49179-afe0-49bb-86d6-87aa62a38bab None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:14:41.159885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef49179-afe0-49bb-86d6-87aa62a38bab None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:14:41.166085 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf859f-92ea-4ea4-bcb3-e08e1e34c05f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:14:41.166728 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf859f-92ea-4ea4-bcb3-e08e1e34c05f service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:41.167008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf859f-92ea-4ea4-bcb3-e08e1e34c05f service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:41.167790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf859f-92ea-4ea4-bcb3-e08e1e34c05f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:41.175708 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf859f-92ea-4ea4-bcb3-e08e1e34c05f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:14:41.177272 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf859f-92ea-4ea4-bcb3-e08e1e34c05f service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:14:41.186882 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 717/1428] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:14:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:14:42.032406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5cc0c-3fa4-45ff-8079-c1b3e39eedf6 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:42.032992 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5cc0c-3fa4-45ff-8079-c1b3e39eedf6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:42.032992 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5cc0c-3fa4-45ff-8079-c1b3e39eedf6 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:42.034333 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 14:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:14:42.039860 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e11c7-4a8a-4f5c-8628-dddf97c9dcaf None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:42.040292 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e11c7-4a8a-4f5c-8628-dddf97c9dcaf None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:42.040414 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e11c7-4a8a-4f5c-8628-dddf97c9dcaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:42.058150 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-f74e11c7-4a8a-4f5c-8628-dddf97c9dcaf None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:14:42.059388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f74e11c7-4a8a-4f5c-8628-dddf97c9dcaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:14:42.067446 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 14:14:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:14:42.217994 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5cc0c-3fa4-45ff-8079-c1b3e39eedf6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:14:42.232266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5cc0c-3fa4-45ff-8079-c1b3e39eedf6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:14:42.237956 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458d379-e502-4beb-8348-7c87eb58bd9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:14:42.238590 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458d379-e502-4beb-8348-7c87eb58bd9a service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:42.238875 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458d379-e502-4beb-8348-7c87eb58bd9a service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:42.239147 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458d379-e502-4beb-8348-7c87eb58bd9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:42.247501 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d458d379-e502-4beb-8348-7c87eb58bd9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:14:42.249137 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d458d379-e502-4beb-8348-7c87eb58bd9a service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:14:42.260247 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 713/1431] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:14:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:14:43.116131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab69191-4833-4c5e-862e-3b139ebed2e8 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:43.116736 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab69191-4833-4c5e-862e-3b139ebed2e8 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:43.116736 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab69191-4833-4c5e-862e-3b139ebed2e8 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:43.118069 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 14:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:14:43.123802 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46fb6d-d66e-426e-90a3-e6b170725940 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:43.124234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46fb6d-d66e-426e-90a3-e6b170725940 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:43.124339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46fb6d-d66e-426e-90a3-e6b170725940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:43.141835 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2f46fb6d-d66e-426e-90a3-e6b170725940 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:14:43.143026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2f46fb6d-d66e-426e-90a3-e6b170725940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:14:43.150962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 14:14:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:14:43.302838 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab69191-4833-4c5e-862e-3b139ebed2e8 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:14:43.317120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab69191-4833-4c5e-862e-3b139ebed2e8 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:14:43.322697 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e801e7b-fb76-40c9-af03-44965d197025 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:14:43.323359 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e801e7b-fb76-40c9-af03-44965d197025 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:43.323645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e801e7b-fb76-40c9-af03-44965d197025 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:43.323918 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e801e7b-fb76-40c9-af03-44965d197025 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:43.331857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e801e7b-fb76-40c9-af03-44965d197025 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:14:43.333452 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e801e7b-fb76-40c9-af03-44965d197025 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:14:43.343322 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 720/1434] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:14:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:14:44.202664 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac29e4-ce7a-4831-aaf1-3f6dfeae9ca1 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:44.203163 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac29e4-ce7a-4831-aaf1-3f6dfeae9ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:44.203282 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac29e4-ce7a-4831-aaf1-3f6dfeae9ca1 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:44.204650 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 14:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:14:44.209897 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248b5aa-3999-40fa-95ff-8c014076608e None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:44.210360 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248b5aa-3999-40fa-95ff-8c014076608e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:44.210489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248b5aa-3999-40fa-95ff-8c014076608e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:44.228436 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-b248b5aa-3999-40fa-95ff-8c014076608e None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:14:44.229627 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b248b5aa-3999-40fa-95ff-8c014076608e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:14:44.237593 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 14:14:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:14:44.389403 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac29e4-ce7a-4831-aaf1-3f6dfeae9ca1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:14:44.403584 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac29e4-ce7a-4831-aaf1-3f6dfeae9ca1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:14:44.408913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7cad2-637b-45b8-b18d-d057b097f737 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:14:44.409546 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7cad2-637b-45b8-b18d-d057b097f737 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:44.409829 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7cad2-637b-45b8-b18d-d057b097f737 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:44.410105 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7cad2-637b-45b8-b18d-d057b097f737 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:44.418848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7cad2-637b-45b8-b18d-d057b097f737 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:14:44.420447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7cad2-637b-45b8-b18d-d057b097f737 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:14:44.430352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 716/1437] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:14:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:14:45.288666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ef0e3-6dc7-4798-afbb-f915506e3af4 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:45.289277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ef0e3-6dc7-4798-afbb-f915506e3af4 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:45.289277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ef0e3-6dc7-4798-afbb-f915506e3af4 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:45.290660 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 14:14:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:14:45.295745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b86a1-0dab-4928-b801-6362ad1d94c5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:45.296151 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b86a1-0dab-4928-b801-6362ad1d94c5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:45.296371 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b86a1-0dab-4928-b801-6362ad1d94c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:45.315462 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-691b86a1-0dab-4928-b801-6362ad1d94c5 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:14:45.316667 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-691b86a1-0dab-4928-b801-6362ad1d94c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:14:45.324577 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 14:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:14:45.476339 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ef0e3-6dc7-4798-afbb-f915506e3af4 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:14:45.491513 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ef0e3-6dc7-4798-afbb-f915506e3af4 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:14:45.496867 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc9363-cab8-4095-97c2-33b30a5ae0ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:14:45.497511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc9363-cab8-4095-97c2-33b30a5ae0ed service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:45.497795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc9363-cab8-4095-97c2-33b30a5ae0ed service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:45.498074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc9363-cab8-4095-97c2-33b30a5ae0ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:45.506085 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffc9363-cab8-4095-97c2-33b30a5ae0ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:14:45.507737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffc9363-cab8-4095-97c2-33b30a5ae0ed service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:14:45.517001 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 723/1440] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:14:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:14:47.632853 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c9bde-2b68-4792-b395-72baabd0ff0b None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:47.633499 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c9bde-2b68-4792-b395-72baabd0ff0b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:47.633499 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c9bde-2b68-4792-b395-72baabd0ff0b None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:47.634767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:14:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:14:47.640021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de525667-d617-44e9-8b82-a50b6d83952b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:47.640474 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de525667-d617-44e9-8b82-a50b6d83952b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:47.640580 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de525667-d617-44e9-8b82-a50b6d83952b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:47.659351 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-de525667-d617-44e9-8b82-a50b6d83952b None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:14:47.660528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-de525667-d617-44e9-8b82-a50b6d83952b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:14:47.670037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:14:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:14:47.708524 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c9bde-2b68-4792-b395-72baabd0ff0b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:14:47.710863 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-479c9bde-2b68-4792-b395-72baabd0ff0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:14:47.723554 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c9bde-2b68-4792-b395-72baabd0ff0b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:14:47.727056 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1baf5-3ca5-4be9-8960-dce8fee4fa35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:14:47.727725 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1baf5-3ca5-4be9-8960-dce8fee4fa35 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:47.728007 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1baf5-3ca5-4be9-8960-dce8fee4fa35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:47.728281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1baf5-3ca5-4be9-8960-dce8fee4fa35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:47.737287 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1baf5-3ca5-4be9-8960-dce8fee4fa35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:14:47.738694 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1baf5-3ca5-4be9-8960-dce8fee4fa35 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:14:47.748062 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 719/1443] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:14:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:14:50.179375 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25faa0-f58d-4a65-8ba0-a598ddbabc14 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:50.179828 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25faa0-f58d-4a65-8ba0-a598ddbabc14 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:50.179923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25faa0-f58d-4a65-8ba0-a598ddbabc14 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:50.181300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:14:50.186526 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1e162-98b2-4db7-80ee-463704efefbb None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:50.186911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1e162-98b2-4db7-80ee-463704efefbb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:50.187075 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1e162-98b2-4db7-80ee-463704efefbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:50.204402 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f0b1e162-98b2-4db7-80ee-463704efefbb None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:14:50.205582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f0b1e162-98b2-4db7-80ee-463704efefbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:14:50.214060 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:14:50.252555 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25faa0-f58d-4a65-8ba0-a598ddbabc14 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:14:50.263563 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25faa0-f58d-4a65-8ba0-a598ddbabc14 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:14:50.266974 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115d99f-9b65-4ac1-a216-3b7211d1de26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:14:50.267635 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115d99f-9b65-4ac1-a216-3b7211d1de26 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:50.267915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115d99f-9b65-4ac1-a216-3b7211d1de26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:50.268186 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115d99f-9b65-4ac1-a216-3b7211d1de26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:50.277709 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2115d99f-9b65-4ac1-a216-3b7211d1de26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:14:50.279125 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2115d99f-9b65-4ac1-a216-3b7211d1de26 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:14:50.289010 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 726/1446] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:14:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:14:52.890411 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd9ab8-e8c8-4304-8b02-09138bd84324 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:52.890951 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd9ab8-e8c8-4304-8b02-09138bd84324 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:52.890951 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd9ab8-e8c8-4304-8b02-09138bd84324 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:52.892379 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:14:52.898614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68720a8-dd78-4b8a-ba85-fb0aed91e0e3 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:52.899055 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68720a8-dd78-4b8a-ba85-fb0aed91e0e3 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:52.899176 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68720a8-dd78-4b8a-ba85-fb0aed91e0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:52.916655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a68720a8-dd78-4b8a-ba85-fb0aed91e0e3 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:14:52.917827 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a68720a8-dd78-4b8a-ba85-fb0aed91e0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:14:52.925930 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:14:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:14:52.964670 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd9ab8-e8c8-4304-8b02-09138bd84324 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:14:52.975178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd9ab8-e8c8-4304-8b02-09138bd84324 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:14:52.978680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f71b6-f64c-4cab-9640-b35f2904c8aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:14:52.979359 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f71b6-f64c-4cab-9640-b35f2904c8aa service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:52.979642 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f71b6-f64c-4cab-9640-b35f2904c8aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:52.979912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f71b6-f64c-4cab-9640-b35f2904c8aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:52.988925 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f71b6-f64c-4cab-9640-b35f2904c8aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:14:52.990414 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f71b6-f64c-4cab-9640-b35f2904c8aa service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:14:52.999797 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 722/1449] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:14:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:14:55.388358 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbfb26-d61b-4836-b435-d0953e1e3aa1 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:55.388958 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbfb26-d61b-4836-b435-d0953e1e3aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:55.388958 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbfb26-d61b-4836-b435-d0953e1e3aa1 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:55.390286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:14:55.395620 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a2bf6-4aec-403c-990b-9dad9e54db44 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:55.396040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a2bf6-4aec-403c-990b-9dad9e54db44 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:55.396193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a2bf6-4aec-403c-990b-9dad9e54db44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:55.413820 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e00a2bf6-4aec-403c-990b-9dad9e54db44 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:14:55.415004 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e00a2bf6-4aec-403c-990b-9dad9e54db44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:14:55.422644 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:14:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:14:55.463486 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfbfb26-d61b-4836-b435-d0953e1e3aa1 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:14:55.474153 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfbfb26-d61b-4836-b435-d0953e1e3aa1 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:14:55.478792 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a35b2e-385d-4533-ab52-3c6afc60a6e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:14:55.479455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a35b2e-385d-4533-ab52-3c6afc60a6e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:55.479739 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a35b2e-385d-4533-ab52-3c6afc60a6e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:55.480019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a35b2e-385d-4533-ab52-3c6afc60a6e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:55.489071 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a35b2e-385d-4533-ab52-3c6afc60a6e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:14:55.490613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a35b2e-385d-4533-ab52-3c6afc60a6e6 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:14:55.499877 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 729/1452] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:14:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:14:57.903270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa1573-f387-4445-9e2a-d6478fb9d04d None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:57.903847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa1573-f387-4445-9e2a-d6478fb9d04d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:57.903847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa1573-f387-4445-9e2a-d6478fb9d04d None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:57.905213 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:14:57.910461 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214bfbe-f26b-4b38-92e8-cd8583e48af3 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:57.910866 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214bfbe-f26b-4b38-92e8-cd8583e48af3 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:57.911019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214bfbe-f26b-4b38-92e8-cd8583e48af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:57.929615 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-2214bfbe-f26b-4b38-92e8-cd8583e48af3 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:14:57.930811 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2214bfbe-f26b-4b38-92e8-cd8583e48af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:14:57.938477 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:14:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:14:57.976176 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa1573-f387-4445-9e2a-d6478fb9d04d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:14:57.986912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa1573-f387-4445-9e2a-d6478fb9d04d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:14:57.990899 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d21cf2-b3b7-4cee-a6d2-a4c1d49c4be5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:14:57.991551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d21cf2-b3b7-4cee-a6d2-a4c1d49c4be5 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:14:57.991829 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d21cf2-b3b7-4cee-a6d2-a4c1d49c4be5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:14:57.992099 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d21cf2-b3b7-4cee-a6d2-a4c1d49c4be5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:14:58.000807 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d21cf2-b3b7-4cee-a6d2-a4c1d49c4be5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:14:58.002209 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d21cf2-b3b7-4cee-a6d2-a4c1d49c4be5 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:14:58.011995 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 725/1455] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:14:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:15:00.298731 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8043ac-8402-46b7-8516-6287f8ed8997 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:00.299313 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8043ac-8402-46b7-8516-6287f8ed8997 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:00.299313 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8043ac-8402-46b7-8516-6287f8ed8997 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:00.300710 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:15:00.306275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f1051-8f55-48cb-9caa-28691736a201 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:00.306699 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f1051-8f55-48cb-9caa-28691736a201 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:00.306898 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f1051-8f55-48cb-9caa-28691736a201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:00.324748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-187f1051-8f55-48cb-9caa-28691736a201 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:15:00.325934 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-187f1051-8f55-48cb-9caa-28691736a201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:15:00.333720 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:15:00.372105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8043ac-8402-46b7-8516-6287f8ed8997 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:15:00.383206 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8043ac-8402-46b7-8516-6287f8ed8997 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:15:00.386638 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41e222-e430-4ebf-ac19-18cdd41a2354 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:15:00.387300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41e222-e430-4ebf-ac19-18cdd41a2354 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:00.387585 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41e222-e430-4ebf-ac19-18cdd41a2354 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:00.387858 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41e222-e430-4ebf-ac19-18cdd41a2354 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:00.397150 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be41e222-e430-4ebf-ac19-18cdd41a2354 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:15:00.398561 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be41e222-e430-4ebf-ac19-18cdd41a2354 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:15:00.407820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 732/1458] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:15:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:15:02.734144 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad241e-bad4-490a-b306-0518ce1d96be None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:02.734735 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad241e-bad4-490a-b306-0518ce1d96be None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:02.734735 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad241e-bad4-490a-b306-0518ce1d96be None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:02.736396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:15:02.742424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f799b2a-baf2-467f-8ca3-6eab738b75b7 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:02.742854 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f799b2a-baf2-467f-8ca3-6eab738b75b7 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:02.742967 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f799b2a-baf2-467f-8ca3-6eab738b75b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:02.749075 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=70 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:15:02.762188 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-2f799b2a-baf2-467f-8ca3-6eab738b75b7 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:15:02.763427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2f799b2a-baf2-467f-8ca3-6eab738b75b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:15:02.771802 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:15:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:15:02.810074 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad241e-bad4-490a-b306-0518ce1d96be None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:15:02.815325 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=73 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:15:02.822149 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad241e-bad4-490a-b306-0518ce1d96be None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:15:02.825594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09219b43-c6d9-49e5-9339-a88b91728559 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:15:02.826250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09219b43-c6d9-49e5-9339-a88b91728559 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:02.826526 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09219b43-c6d9-49e5-9339-a88b91728559 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:02.826795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09219b43-c6d9-49e5-9339-a88b91728559 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:02.835806 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09219b43-c6d9-49e5-9339-a88b91728559 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:15:02.837220 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09219b43-c6d9-49e5-9339-a88b91728559 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:15:02.846269 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 728/1461] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:15:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:15:05.237166 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e106f-77a8-4f96-b615-2c8e0bdcf7bb None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:05.237664 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e106f-77a8-4f96-b615-2c8e0bdcf7bb None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:05.237793 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e106f-77a8-4f96-b615-2c8e0bdcf7bb None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:05.239110 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:15:05.244591 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac4377-85b2-445a-a568-0b3e00a960c8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:05.245037 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac4377-85b2-445a-a568-0b3e00a960c8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:05.245187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac4377-85b2-445a-a568-0b3e00a960c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:05.263380 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1fac4377-85b2-445a-a568-0b3e00a960c8 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:15:05.264558 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1fac4377-85b2-445a-a568-0b3e00a960c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:15:05.272923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:15:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:15:05.310722 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e106f-77a8-4f96-b615-2c8e0bdcf7bb None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:15:05.321639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e106f-77a8-4f96-b615-2c8e0bdcf7bb None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:15:05.325165 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36cb4ab-78b2-4184-a72a-28d95000c49b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:15:05.325798 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cb4ab-78b2-4184-a72a-28d95000c49b service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:05.326080 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cb4ab-78b2-4184-a72a-28d95000c49b service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:05.326353 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cb4ab-78b2-4184-a72a-28d95000c49b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:05.335809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36cb4ab-78b2-4184-a72a-28d95000c49b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:15:05.337227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36cb4ab-78b2-4184-a72a-28d95000c49b service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:15:05.346491 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 735/1464] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:15:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:15:07.666756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8400bc-1220-42a0-ab26-1e20180fd46f None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:07.667353 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8400bc-1220-42a0-ab26-1e20180fd46f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:07.667353 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8400bc-1220-42a0-ab26-1e20180fd46f None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:07.668711 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:15:07.674104 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597f3c9-a94f-4d37-8d08-5a1d9c323df1 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:07.674552 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597f3c9-a94f-4d37-8d08-5a1d9c323df1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:07.674673 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597f3c9-a94f-4d37-8d08-5a1d9c323df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:07.693227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-9597f3c9-a94f-4d37-8d08-5a1d9c323df1 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:15:07.694573 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9597f3c9-a94f-4d37-8d08-5a1d9c323df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:15:07.702201 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:15:07.739148 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8400bc-1220-42a0-ab26-1e20180fd46f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:15:07.750638 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8400bc-1220-42a0-ab26-1e20180fd46f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:15:07.754061 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1441c6b-d9f3-4d2a-b556-e87a5ca9c269 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:15:07.754695 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1441c6b-d9f3-4d2a-b556-e87a5ca9c269 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:07.754977 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1441c6b-d9f3-4d2a-b556-e87a5ca9c269 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:07.756001 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1441c6b-d9f3-4d2a-b556-e87a5ca9c269 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:07.764724 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1441c6b-d9f3-4d2a-b556-e87a5ca9c269 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:15:07.766129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1441c6b-d9f3-4d2a-b556-e87a5ca9c269 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:15:07.776301 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 731/1467] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:15:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:15:10.053899 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceef83f-0565-47a0-8207-de92767bf7cd None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:10.054446 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceef83f-0565-47a0-8207-de92767bf7cd None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:10.054446 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceef83f-0565-47a0-8207-de92767bf7cd None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:10.055867 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:15:10.061057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e6053-8e99-4403-8e73-9e15132b3b14 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:10.061506 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e6053-8e99-4403-8e73-9e15132b3b14 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:10.061615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e6053-8e99-4403-8e73-9e15132b3b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:10.079615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b66e6053-8e99-4403-8e73-9e15132b3b14 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:15:10.080789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b66e6053-8e99-4403-8e73-9e15132b3b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:15:10.089312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:15:10.125664 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceef83f-0565-47a0-8207-de92767bf7cd None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:15:10.137332 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceef83f-0565-47a0-8207-de92767bf7cd None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:15:10.141024 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf539a77-ce34-40e6-926f-358aed9cb32e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:15:10.141666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf539a77-ce34-40e6-926f-358aed9cb32e service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:10.141949 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf539a77-ce34-40e6-926f-358aed9cb32e service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:10.142227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf539a77-ce34-40e6-926f-358aed9cb32e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:10.151903 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf539a77-ce34-40e6-926f-358aed9cb32e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:15:10.153338 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf539a77-ce34-40e6-926f-358aed9cb32e service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:15:10.163113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 738/1470] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:15:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:15:20.074505 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=17 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:15:20.158676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=17 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:15:42.360282 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604f31f-d81a-4e1d-94e2-da0b40424eca None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:42.360740 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604f31f-d81a-4e1d-94e2-da0b40424eca None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:42.360845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604f31f-d81a-4e1d-94e2-da0b40424eca None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:42.362221 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 733/1471] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:15:42.367978 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106abc1-0e61-4457-9e1b-3ab2c572414a None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:42.368403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106abc1-0e61-4457-9e1b-3ab2c572414a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:42.368540 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106abc1-0e61-4457-9e1b-3ab2c572414a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:42.386235 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-0106abc1-0e61-4457-9e1b-3ab2c572414a None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:15:42.387484 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0106abc1-0e61-4457-9e1b-3ab2c572414a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:15:42.395396 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 739/1472] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:15:42.442857 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be420853-3838-440e-ac75-7ff05ff55311 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:42.443142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be420853-3838-440e-ac75-7ff05ff55311 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:42.443451 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be420853-3838-440e-ac75-7ff05ff55311 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:42.444826 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 734/1473] 10.222.0.81 () {54 vars in 962 bytes} [Tue Jun 7 14:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:15:42.449747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff4b7a-1985-4ef6-a791-5b616e16caea None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:42.450039 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff4b7a-1985-4ef6-a791-5b616e16caea None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:42.450322 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff4b7a-1985-4ef6-a791-5b616e16caea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:42.468032 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6dff4b7a-1985-4ef6-a791-5b616e16caea None None] MFA Rules not processed for user `b6180393da0f495db964f0b2588d8b90`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:15:42.469223 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6dff4b7a-1985-4ef6-a791-5b616e16caea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:15:42.476282 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 740/1474] 10.222.0.81 () {58 vars in 1059 bytes} [Tue Jun 7 14:15:42 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:15:42.483194 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be420853-3838-440e-ac75-7ff05ff55311 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:15:42.494509 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be420853-3838-440e-ac75-7ff05ff55311 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:15:42.497958 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce2fe1-9c51-4e78-95b5-8a3cd1796630 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6180393da0f495db964f0b2588d8b90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:15:42.498580 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce2fe1-9c51-4e78-95b5-8a3cd1796630 service glance] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:42.498857 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce2fe1-9c51-4e78-95b5-8a3cd1796630 service glance] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:42.499127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce2fe1-9c51-4e78-95b5-8a3cd1796630 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:42.509846 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce2fe1-9c51-4e78-95b5-8a3cd1796630 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:15:42.511295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce2fe1-9c51-4e78-95b5-8a3cd1796630 service glance] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:15:42.521330 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 735/1475] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:15:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:15:44.871859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67281977-25c5-40cd-81d7-d9bfd68513ca None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:44.872426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67281977-25c5-40cd-81d7-d9bfd68513ca None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:44.872426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67281977-25c5-40cd-81d7-d9bfd68513ca None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:44.873818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 741/1476] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:15:44.879609 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c055a-92af-4020-a19e-3e9c78fcc464 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:44.880005 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c055a-92af-4020-a19e-3e9c78fcc464 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:44.880159 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c055a-92af-4020-a19e-3e9c78fcc464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:44.897551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-874c055a-92af-4020-a19e-3e9c78fcc464 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:15:44.898679 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-874c055a-92af-4020-a19e-3e9c78fcc464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:15:44.906776 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 736/1477] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:15:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:15:44.931000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67281977-25c5-40cd-81d7-d9bfd68513ca None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:15:44.944216 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67281977-25c5-40cd-81d7-d9bfd68513ca None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:15:44.948628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b4032-41f9-4aab-8744-8eed5c7d84df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:15:44.949276 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b4032-41f9-4aab-8744-8eed5c7d84df admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:44.949561 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b4032-41f9-4aab-8744-8eed5c7d84df admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:44.949835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b4032-41f9-4aab-8744-8eed5c7d84df admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:44.950572 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b4032-41f9-4aab-8744-8eed5c7d84df admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:15:44.952339 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: WARNING py.warnings [None req-902b4032-41f9-4aab-8744-8eed5c7d84df admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:15:44.952339 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: warnings.warn(msg) Jun 07 14:15:44.952339 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: Jun 07 14:15:44.953485 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b4032-41f9-4aab-8744-8eed5c7d84df admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:15:44.973191 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 742/1478] 10.222.0.81 () {56 vars in 1108 bytes} [Tue Jun 7 14:15:44 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:15:47.259106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77b443-0b6f-43c3-a048-96a7024641d3 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:47.259702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77b443-0b6f-43c3-a048-96a7024641d3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:47.259702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77b443-0b6f-43c3-a048-96a7024641d3 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:47.261061 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 737/1479] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:15:47.266173 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85917d-9af0-41c8-b403-af6a8ace622a None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:47.266625 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85917d-9af0-41c8-b403-af6a8ace622a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:47.266732 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85917d-9af0-41c8-b403-af6a8ace622a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:47.285925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-bb85917d-9af0-41c8-b403-af6a8ace622a None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:15:47.287275 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-bb85917d-9af0-41c8-b403-af6a8ace622a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:15:47.294883 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 743/1480] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:15:47.338278 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77b443-0b6f-43c3-a048-96a7024641d3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:15:47.352935 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77b443-0b6f-43c3-a048-96a7024641d3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:15:47.358151 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee7977-6cde-47fe-bf48-2167a870a225 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:15:47.358781 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee7977-6cde-47fe-bf48-2167a870a225 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:47.359056 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee7977-6cde-47fe-bf48-2167a870a225 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:47.359353 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee7977-6cde-47fe-bf48-2167a870a225 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:47.367867 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee7977-6cde-47fe-bf48-2167a870a225 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:15:47.369426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee7977-6cde-47fe-bf48-2167a870a225 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:15:47.379157 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 738/1481] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:15:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:15:49.802730 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970159b-6681-4076-b0fb-33aa25e53dfb None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:49.803288 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970159b-6681-4076-b0fb-33aa25e53dfb None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:49.803288 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970159b-6681-4076-b0fb-33aa25e53dfb None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:49.804613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 744/1482] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:15:49.810027 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798dec13-63c5-41d0-ad87-22a7180b3da1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:49.810437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798dec13-63c5-41d0-ad87-22a7180b3da1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:49.810587 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798dec13-63c5-41d0-ad87-22a7180b3da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:49.828509 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-798dec13-63c5-41d0-ad87-22a7180b3da1 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:15:49.829706 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-798dec13-63c5-41d0-ad87-22a7180b3da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:15:49.838404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 739/1483] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:15:49.874914 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970159b-6681-4076-b0fb-33aa25e53dfb None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:15:49.886103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970159b-6681-4076-b0fb-33aa25e53dfb None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:15:49.889524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42762f13-c0ea-42fd-b6a5-e34099d3890e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:15:49.890155 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42762f13-c0ea-42fd-b6a5-e34099d3890e service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:49.890432 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42762f13-c0ea-42fd-b6a5-e34099d3890e service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:49.890703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42762f13-c0ea-42fd-b6a5-e34099d3890e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:49.899822 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42762f13-c0ea-42fd-b6a5-e34099d3890e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:15:49.901220 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42762f13-c0ea-42fd-b6a5-e34099d3890e service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:15:49.911038 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 745/1484] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:15:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:15:52.229052 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab50181-67e2-4bc4-b130-e3020ffe438c None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:52.229572 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab50181-67e2-4bc4-b130-e3020ffe438c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:52.229712 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab50181-67e2-4bc4-b130-e3020ffe438c None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:52.230987 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 740/1485] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:15:52.236401 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da14f69-e5c6-4216-a352-bf856ecc3547 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:52.236837 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da14f69-e5c6-4216-a352-bf856ecc3547 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:52.236935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da14f69-e5c6-4216-a352-bf856ecc3547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:52.255195 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-9da14f69-e5c6-4216-a352-bf856ecc3547 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:15:52.256401 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9da14f69-e5c6-4216-a352-bf856ecc3547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:15:52.264009 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 746/1486] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:15:52.302723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab50181-67e2-4bc4-b130-e3020ffe438c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:15:52.313148 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab50181-67e2-4bc4-b130-e3020ffe438c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:15:52.317300 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85193cf-bced-40f2-b81f-9d3b5cb93994 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:15:52.317928 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85193cf-bced-40f2-b81f-9d3b5cb93994 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:52.318209 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85193cf-bced-40f2-b81f-9d3b5cb93994 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:52.318479 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85193cf-bced-40f2-b81f-9d3b5cb93994 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:52.327333 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85193cf-bced-40f2-b81f-9d3b5cb93994 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:15:52.328753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85193cf-bced-40f2-b81f-9d3b5cb93994 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:15:52.337980 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 741/1487] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:15:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:15:54.755515 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480214b-fbba-4a92-bcf9-d28c651c852d None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:54.756172 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480214b-fbba-4a92-bcf9-d28c651c852d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:54.756172 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480214b-fbba-4a92-bcf9-d28c651c852d None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:54.757436 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 747/1488] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:15:54.762812 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c8319-91e7-4976-bb0b-926abc0a72c4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:54.763257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c8319-91e7-4976-bb0b-926abc0a72c4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:54.763407 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c8319-91e7-4976-bb0b-926abc0a72c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:54.781647 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9a4c8319-91e7-4976-bb0b-926abc0a72c4 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:15:54.782843 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9a4c8319-91e7-4976-bb0b-926abc0a72c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:15:54.791171 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 742/1489] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:15:54.831091 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4480214b-fbba-4a92-bcf9-d28c651c852d None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:15:54.842329 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4480214b-fbba-4a92-bcf9-d28c651c852d None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:15:54.845773 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe28368-5771-486a-a15c-dc9a21cc4f46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:15:54.846404 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe28368-5771-486a-a15c-dc9a21cc4f46 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:54.846682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe28368-5771-486a-a15c-dc9a21cc4f46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:54.846953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe28368-5771-486a-a15c-dc9a21cc4f46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:54.856494 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe28368-5771-486a-a15c-dc9a21cc4f46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:15:54.857897 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe28368-5771-486a-a15c-dc9a21cc4f46 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:15:54.867645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 748/1490] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:15:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:15:57.375441 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada6c10-aab3-4056-a59f-b34213f53871 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:57.375998 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada6c10-aab3-4056-a59f-b34213f53871 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:57.375998 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada6c10-aab3-4056-a59f-b34213f53871 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:57.377387 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 743/1491] 10.222.0.81 () {54 vars in 926 bytes} [Tue Jun 7 14:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:15:57.383457 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c03e8fb-af95-42fe-a14f-544491296e63 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:57.383877 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c03e8fb-af95-42fe-a14f-544491296e63 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:57.384006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c03e8fb-af95-42fe-a14f-544491296e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:57.402643 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-1c03e8fb-af95-42fe-a14f-544491296e63 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:15:57.403871 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1c03e8fb-af95-42fe-a14f-544491296e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:15:57.412060 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 749/1492] 10.222.0.81 () {58 vars in 1023 bytes} [Tue Jun 7 14:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:15:57.450506 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada6c10-aab3-4056-a59f-b34213f53871 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:15:57.461912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada6c10-aab3-4056-a59f-b34213f53871 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:15:57.465500 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b5c7a-34d6-46ac-a195-8df6517c57ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:15:57.466138 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b5c7a-34d6-46ac-a195-8df6517c57ef service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:15:57.466419 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b5c7a-34d6-46ac-a195-8df6517c57ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:15:57.466698 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b5c7a-34d6-46ac-a195-8df6517c57ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:15:57.475610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b5c7a-34d6-46ac-a195-8df6517c57ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:15:57.477024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b5c7a-34d6-46ac-a195-8df6517c57ef service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:15:57.486528 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 744/1493] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:15:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:16:07.397633 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=33 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:16:07.482085 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=33 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:17:32.412086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a7fb0-c513-4e81-a17f-c6d5b31b26a7 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:17:32.412674 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a7fb0-c513-4e81-a17f-c6d5b31b26a7 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:17:32.412674 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a7fb0-c513-4e81-a17f-c6d5b31b26a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:17:32.431352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-5b7a7fb0-c513-4e81-a17f-c6d5b31b26a7 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:17:32.432554 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5b7a7fb0-c513-4e81-a17f-c6d5b31b26a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:17:32.440512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 750/1494] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:17:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:17:32.448276 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdade1-ce7c-404d-bf98-d57fee4eab2e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:17:32.448770 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdade1-ce7c-404d-bf98-d57fee4eab2e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:17:32.448890 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdade1-ce7c-404d-bf98-d57fee4eab2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:17:32.466486 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-25bdade1-ce7c-404d-bf98-d57fee4eab2e None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:17:32.467706 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-25bdade1-ce7c-404d-bf98-d57fee4eab2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:17:32.476060 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 745/1495] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:17:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:17:32.480369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a7fb0-c513-4e81-a17f-c6d5b31b26a7 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:17:32.490853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a7fb0-c513-4e81-a17f-c6d5b31b26a7 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:17:32.495425 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a48b2-29ab-4dae-bd42-23066c382382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:17:32.496078 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a48b2-29ab-4dae-bd42-23066c382382 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:17:32.496357 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a48b2-29ab-4dae-bd42-23066c382382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:17:32.496631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a48b2-29ab-4dae-bd42-23066c382382 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:17:32.497351 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a48b2-29ab-4dae-bd42-23066c382382 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:17:32.497653 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a48b2-29ab-4dae-bd42-23066c382382 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:17:32.498644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: WARNING py.warnings [None req-650a48b2-29ab-4dae-bd42-23066c382382 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:17:32.498644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: warnings.warn(msg) Jun 07 14:17:32.498644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: Jun 07 14:17:32.499568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a48b2-29ab-4dae-bd42-23066c382382 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:17:32.511028 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 751/1496] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:17:32.519104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdade1-ce7c-404d-bf98-d57fee4eab2e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:17:32.528146 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdade1-ce7c-404d-bf98-d57fee4eab2e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:17:32.533338 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c8dfa-c16a-431e-8ed6-5c897954bc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:17:32.533990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c8dfa-c16a-431e-8ed6-5c897954bc06 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:17:32.534273 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c8dfa-c16a-431e-8ed6-5c897954bc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:17:32.534545 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c8dfa-c16a-431e-8ed6-5c897954bc06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:17:32.535234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c8dfa-c16a-431e-8ed6-5c897954bc06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:17:32.536955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-cc2c8dfa-c16a-431e-8ed6-5c897954bc06 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:17:32.536955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: warnings.warn(msg) Jun 07 14:17:32.536955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: Jun 07 14:17:32.538146 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c8dfa-c16a-431e-8ed6-5c897954bc06 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:17:32.574886 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 746/1497] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:17:32 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:17:32.579104 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a48b2-29ab-4dae-bd42-23066c382382 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:17:32.588984 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a48b2-29ab-4dae-bd42-23066c382382 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:17:32.644692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726530fe-0c61-4242-ad00-86472e9047c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:17:32.645357 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726530fe-0c61-4242-ad00-86472e9047c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:17:32.645636 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726530fe-0c61-4242-ad00-86472e9047c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:17:32.645910 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726530fe-0c61-4242-ad00-86472e9047c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:17:32.646590 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726530fe-0c61-4242-ad00-86472e9047c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:17:32.647610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: WARNING py.warnings [None req-726530fe-0c61-4242-ad00-86472e9047c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:17:32.647610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: warnings.warn(msg) Jun 07 14:17:32.647610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: Jun 07 14:17:32.648811 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726530fe-0c61-4242-ad00-86472e9047c5 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:17:32.670796 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 752/1498] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:17:32 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:17:32.675510 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c8dfa-c16a-431e-8ed6-5c897954bc06 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:17:32.685409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c8dfa-c16a-431e-8ed6-5c897954bc06 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:17:32.690239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ba14f-7407-480e-a0dd-37131ddb37c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:17:32.690908 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ba14f-7407-480e-a0dd-37131ddb37c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:17:32.691195 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ba14f-7407-480e-a0dd-37131ddb37c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:17:32.691510 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ba14f-7407-480e-a0dd-37131ddb37c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:17:32.692245 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ba14f-7407-480e-a0dd-37131ddb37c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:17:32.693257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-a42ba14f-7407-480e-a0dd-37131ddb37c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:17:32.693257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: warnings.warn(msg) Jun 07 14:17:32.693257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: Jun 07 14:17:32.694207 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ba14f-7407-480e-a0dd-37131ddb37c7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:17:32.706864 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 747/1499] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:17:32.710668 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726530fe-0c61-4242-ad00-86472e9047c5 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:17:32.720844 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726530fe-0c61-4242-ad00-86472e9047c5 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:17:32.725400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847d08a-3682-4ac8-9f1f-abdaf3c203aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:17:32.726054 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847d08a-3682-4ac8-9f1f-abdaf3c203aa admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:17:32.726367 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847d08a-3682-4ac8-9f1f-abdaf3c203aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:17:32.726645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847d08a-3682-4ac8-9f1f-abdaf3c203aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:17:32.727400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0847d08a-3682-4ac8-9f1f-abdaf3c203aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:17:32.728397 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: WARNING py.warnings [None req-0847d08a-3682-4ac8-9f1f-abdaf3c203aa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:17:32.728397 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: warnings.warn(msg) Jun 07 14:17:32.728397 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: Jun 07 14:17:32.729290 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0847d08a-3682-4ac8-9f1f-abdaf3c203aa admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:17:32.743442 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 753/1500] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:17:32.747617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ba14f-7407-480e-a0dd-37131ddb37c7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:17:32.757362 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ba14f-7407-480e-a0dd-37131ddb37c7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:17:32.761827 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f1e47-728c-4a1d-a1e9-2b158f754013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:17:32.762699 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f1e47-728c-4a1d-a1e9-2b158f754013 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:17:32.762986 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f1e47-728c-4a1d-a1e9-2b158f754013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:17:32.763298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f1e47-728c-4a1d-a1e9-2b158f754013 admin admin] PATH_INFO: `/v3/projects/45e55a82615145a8830cb4aa3903ea9a/users/5cef00459f3c4a968afef99e6d171cfa/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:17:32.778660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f1e47-728c-4a1d-a1e9-2b158f754013 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45e55a82615145a8830cb4aa3903ea9a, user_id=5cef00459f3c4a968afef99e6d171cfa, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:17:32.779706 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-ab5f1e47-728c-4a1d-a1e9-2b158f754013 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:17:32.779706 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: warnings.warn(msg) Jun 07 14:17:32.779706 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: Jun 07 14:17:32.782233 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f1e47-728c-4a1d-a1e9-2b158f754013 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:17:32.794698 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 748/1501] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:17:32 2022] PUT /identity/v3/projects/45e55a82615145a8830cb4aa3903ea9a/users/5cef00459f3c4a968afef99e6d171cfa/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:17:32.806712 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f8e89-d63a-4406-b187-222c422c4457 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:17:32.807007 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f8e89-d63a-4406-b187-222c422c4457 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:17:32.807345 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f8e89-d63a-4406-b187-222c422c4457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:17:32.833757 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-957f8e89-d63a-4406-b187-222c422c4457 None None] MFA Rules not processed for user `5cef00459f3c4a968afef99e6d171cfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:17:32.835039 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-957f8e89-d63a-4406-b187-222c422c4457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:17:32.901882 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 754/1502] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:17:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:17:32.915555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7dae22-3836-4e21-bb7c-ab05598bb8ff None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:17:32.915848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7dae22-3836-4e21-bb7c-ab05598bb8ff None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:17:32.916132 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7dae22-3836-4e21-bb7c-ab05598bb8ff None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:17:32.917542 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 749/1503] 10.222.0.81 () {54 vars in 963 bytes} [Tue Jun 7 14:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:17:32.922276 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a81b91-0dc4-4a32-b2b0-9a6c467bd572 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:17:32.922505 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a81b91-0dc4-4a32-b2b0-9a6c467bd572 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:17:32.922785 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a81b91-0dc4-4a32-b2b0-9a6c467bd572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:17:32.940895 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-f9a81b91-0dc4-4a32-b2b0-9a6c467bd572 None None] MFA Rules not processed for user `38b1ff21b8e5445b878740a327e4442f`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:17:32.942311 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f9a81b91-0dc4-4a32-b2b0-9a6c467bd572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:17:32.987191 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:17:32.995618 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 755/1504] 10.222.0.81 () {58 vars in 1060 bytes} [Tue Jun 7 14:17:32 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:17:33.003412 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7dae22-3836-4e21-bb7c-ab05598bb8ff None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:17:33.013678 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7dae22-3836-4e21-bb7c-ab05598bb8ff None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:17:33.017865 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd2acc-5634-4d34-8a4f-05e8c000f669 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:17:33.018504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd2acc-5634-4d34-8a4f-05e8c000f669 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:17:33.018786 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd2acc-5634-4d34-8a4f-05e8c000f669 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:17:33.019060 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd2acc-5634-4d34-8a4f-05e8c000f669 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:17:33.027084 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd2acc-5634-4d34-8a4f-05e8c000f669 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:17:33.028531 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd2acc-5634-4d34-8a4f-05e8c000f669 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:17:33.037741 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 750/1505] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:17:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:17:33.079362 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd40966-c716-468c-8c85-f2400db2ab77 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:17:33.079653 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd40966-c716-468c-8c85-f2400db2ab77 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:17:33.079934 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd40966-c716-468c-8c85-f2400db2ab77 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:17:33.081219 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 756/1506] 10.222.0.81 () {52 vars in 829 bytes} [Tue Jun 7 14:17:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:17:33.084874 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd2acc-5634-4d34-8a4f-05e8c000f669 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:17:33.146911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd2acc-5634-4d34-8a4f-05e8c000f669 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:17:33.151418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce66440-abac-4855-b33c-73186b1de2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:17:33.152291 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce66440-abac-4855-b33c-73186b1de2f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:17:33.152574 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce66440-abac-4855-b33c-73186b1de2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:17:33.152849 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce66440-abac-4855-b33c-73186b1de2f6 admin admin] PATH_INFO: `/v3/users/5cef00459f3c4a968afef99e6d171cfa` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:17:33.154049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce66440-abac-4855-b33c-73186b1de2f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cef00459f3c4a968afef99e6d171cfa)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:17:33.155050 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-bce66440-abac-4855-b33c-73186b1de2f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:17:33.155050 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: warnings.warn(msg) Jun 07 14:17:33.155050 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: Jun 07 14:17:33.156403 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce66440-abac-4855-b33c-73186b1de2f6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:17:33.220236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bce66440-abac-4855-b33c-73186b1de2f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cef00459f3c4a968afef99e6d171cfa'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:17:33.220885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bce66440-abac-4855-b33c-73186b1de2f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cef00459f3c4a968afef99e6d171cfa'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:17:33.221599 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bce66440-abac-4855-b33c-73186b1de2f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cef00459f3c4a968afef99e6d171cfa'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:17:33.233925 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bce66440-abac-4855-b33c-73186b1de2f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cef00459f3c4a968afef99e6d171cfa'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:17:33.251858 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bce66440-abac-4855-b33c-73186b1de2f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cef00459f3c4a968afef99e6d171cfa'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:17:33.255703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:17:33.269439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 751/1507] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:17:33 2022] DELETE /identity/v3/users/5cef00459f3c4a968afef99e6d171cfa => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:17:33.274087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89c3bd-bae3-4980-8423-68295f403aa6 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:17:33.274428 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89c3bd-bae3-4980-8423-68295f403aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:17:33.274711 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89c3bd-bae3-4980-8423-68295f403aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:17:33.293027 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-9a89c3bd-bae3-4980-8423-68295f403aa6 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:17:33.294249 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9a89c3bd-bae3-4980-8423-68295f403aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:17:33.353489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 757/1508] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:17:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:17:33.363258 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce66440-abac-4855-b33c-73186b1de2f6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:17:33.364679 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bce66440-abac-4855-b33c-73186b1de2f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:17:33.419951 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce66440-abac-4855-b33c-73186b1de2f6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:17:33.423868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df982c6-1ef7-444f-8baf-900c3e116489 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:17:33.424533 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df982c6-1ef7-444f-8baf-900c3e116489 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:17:33.424820 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df982c6-1ef7-444f-8baf-900c3e116489 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:17:33.425099 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df982c6-1ef7-444f-8baf-900c3e116489 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:17:33.434061 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df982c6-1ef7-444f-8baf-900c3e116489 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:17:33.435524 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df982c6-1ef7-444f-8baf-900c3e116489 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:17:33.445275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 752/1509] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:17:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:17:33.845729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89c3bd-bae3-4980-8423-68295f403aa6 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:17:33.847308 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9a89c3bd-bae3-4980-8423-68295f403aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:17:33.860033 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89c3bd-bae3-4980-8423-68295f403aa6 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:17:33.864880 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d72071-07fe-4aa0-91f4-075b6f60f9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:17:33.865757 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d72071-07fe-4aa0-91f4-075b6f60f9cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:17:33.866045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d72071-07fe-4aa0-91f4-075b6f60f9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:17:33.866337 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d72071-07fe-4aa0-91f4-075b6f60f9cc admin admin] PATH_INFO: `/v3/projects/45e55a82615145a8830cb4aa3903ea9a` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:17:33.867545 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d72071-07fe-4aa0-91f4-075b6f60f9cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=45e55a82615145a8830cb4aa3903ea9a)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:17:33.868558 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: WARNING py.warnings [None req-87d72071-07fe-4aa0-91f4-075b6f60f9cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:17:33.868558 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: warnings.warn(msg) Jun 07 14:17:33.868558 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: Jun 07 14:17:33.869766 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d72071-07fe-4aa0-91f4-075b6f60f9cc admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:17:33.931140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-87d72071-07fe-4aa0-91f4-075b6f60f9cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45e55a82615145a8830cb4aa3903ea9a', 'request_id': 'req-87d72071-07fe-4aa0-91f4-075b6f60f9cc'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:17:33.941573 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-87d72071-07fe-4aa0-91f4-075b6f60f9cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45e55a82615145a8830cb4aa3903ea9a', 'request_id': 'req-87d72071-07fe-4aa0-91f4-075b6f60f9cc'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:17:33.955604 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-87d72071-07fe-4aa0-91f4-075b6f60f9cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45e55a82615145a8830cb4aa3903ea9a', 'request_id': 'req-87d72071-07fe-4aa0-91f4-075b6f60f9cc'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:17:33.966244 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-87d72071-07fe-4aa0-91f4-075b6f60f9cc admin admin] The token cache is being invalidate because project 45e55a82615145a8830cb4aa3903ea9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:17:33.966530 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-87d72071-07fe-4aa0-91f4-075b6f60f9cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:17:33.967167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-87d72071-07fe-4aa0-91f4-075b6f60f9cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:17:33.968845 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 758/1510] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:17:33 2022] DELETE /identity/v3/projects/45e55a82615145a8830cb4aa3903ea9a => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:17:43.441320 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:17:43.963285 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:13.876391 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756fbd4-72fb-4ae0-a488-d7d3b4cb14b3 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:13.876992 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756fbd4-72fb-4ae0-a488-d7d3b4cb14b3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:13.876992 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756fbd4-72fb-4ae0-a488-d7d3b4cb14b3 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:13.878300 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 14:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:13.883240 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da07c4f-c618-469c-ac9e-d4fbfb803162 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:13.883702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da07c4f-c618-469c-ac9e-d4fbfb803162 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:13.883815 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da07c4f-c618-469c-ac9e-d4fbfb803162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:13.901579 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-4da07c4f-c618-469c-ac9e-d4fbfb803162 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:13.902775 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4da07c4f-c618-469c-ac9e-d4fbfb803162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:13.964441 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 14:19:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:19:14.136358 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756fbd4-72fb-4ae0-a488-d7d3b4cb14b3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:14.138033 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5756fbd4-72fb-4ae0-a488-d7d3b4cb14b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:14.212865 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756fbd4-72fb-4ae0-a488-d7d3b4cb14b3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:14.219005 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a37b16-ad7d-4c5d-baab-40e6b17e01fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:14.219682 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a37b16-ad7d-4c5d-baab-40e6b17e01fd service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:14.219973 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a37b16-ad7d-4c5d-baab-40e6b17e01fd service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:14.220246 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a37b16-ad7d-4c5d-baab-40e6b17e01fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:14.228247 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a37b16-ad7d-4c5d-baab-40e6b17e01fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:14.230077 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a37b16-ad7d-4c5d-baab-40e6b17e01fd service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:14.239439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 754/1513] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:19:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:19:15.132021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c6be6-fd6e-4fe5-aaf7-e4b5b3bf14ff None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:15.132585 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c6be6-fd6e-4fe5-aaf7-e4b5b3bf14ff None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:15.132694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c6be6-fd6e-4fe5-aaf7-e4b5b3bf14ff None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:15.134054 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 14:19:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:15.139445 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf29683-0d1c-4fe2-ad81-c34526062560 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:15.139919 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf29683-0d1c-4fe2-ad81-c34526062560 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:15.140024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf29683-0d1c-4fe2-ad81-c34526062560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:15.158380 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ebf29683-0d1c-4fe2-ad81-c34526062560 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:15.159599 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ebf29683-0d1c-4fe2-ad81-c34526062560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:15.167302 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 14:19:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:19:15.348246 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c6be6-fd6e-4fe5-aaf7-e4b5b3bf14ff None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:15.368138 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c6be6-fd6e-4fe5-aaf7-e4b5b3bf14ff None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:15.373665 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576fb09-db74-4e6b-b009-4368f3c966db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:15.374308 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576fb09-db74-4e6b-b009-4368f3c966db service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:15.374588 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576fb09-db74-4e6b-b009-4368f3c966db service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:15.374886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576fb09-db74-4e6b-b009-4368f3c966db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:15.387605 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576fb09-db74-4e6b-b009-4368f3c966db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:15.389311 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576fb09-db74-4e6b-b009-4368f3c966db service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:15.399032 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 761/1516] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:19:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:19:17.198173 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ba248-b80b-4de7-95b8-a137a67cf3fb None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:17.198639 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ba248-b80b-4de7-95b8-a137a67cf3fb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:17.198738 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ba248-b80b-4de7-95b8-a137a67cf3fb None None] PATH_INFO: `/v3` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:17.200051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 756/1517] 10.222.0.75 () {54 vars in 925 bytes} [Tue Jun 7 14:19:17 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:17.223325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22297c64-ef24-41cf-90af-60fe6e77ba23 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:17.223750 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22297c64-ef24-41cf-90af-60fe6e77ba23 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:17.223855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22297c64-ef24-41cf-90af-60fe6e77ba23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:17.242016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-22297c64-ef24-41cf-90af-60fe6e77ba23 None None] MFA Rules not processed for user `ab73cf3febbf4c768cd0116306f16c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:17.243198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-22297c64-ef24-41cf-90af-60fe6e77ba23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:17.306698 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 762/1518] 10.222.0.75 () {58 vars in 1015 bytes} [Tue Jun 7 14:19:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:19:21.843818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f2d77-1dc1-4208-9cf7-85603e34489b None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:21.844385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f2d77-1dc1-4208-9cf7-85603e34489b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:21.844385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f2d77-1dc1-4208-9cf7-85603e34489b None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:21.845726 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 757/1519] 10.222.0.81 () {54 vars in 960 bytes} [Tue Jun 7 14:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:21.850627 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e844e58-8bb2-463d-8d2b-0fa196d83d32 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:21.851052 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e844e58-8bb2-463d-8d2b-0fa196d83d32 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:21.851186 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e844e58-8bb2-463d-8d2b-0fa196d83d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:21.869607 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6e844e58-8bb2-463d-8d2b-0fa196d83d32 None None] MFA Rules not processed for user `ab73cf3febbf4c768cd0116306f16c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:21.870779 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6e844e58-8bb2-463d-8d2b-0fa196d83d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:21.880116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 763/1520] 10.222.0.81 () {58 vars in 1057 bytes} [Tue Jun 7 14:19:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:19:21.887664 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f2d77-1dc1-4208-9cf7-85603e34489b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:21.901298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f2d77-1dc1-4208-9cf7-85603e34489b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:21.906137 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25ae4d8-e24f-4f4e-9ddb-171c6c705fbb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab73cf3febbf4c768cd0116306f16c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:21.906762 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ae4d8-e24f-4f4e-9ddb-171c6c705fbb service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:21.907068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ae4d8-e24f-4f4e-9ddb-171c6c705fbb service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:21.907352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ae4d8-e24f-4f4e-9ddb-171c6c705fbb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:21.915357 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25ae4d8-e24f-4f4e-9ddb-171c6c705fbb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:21.916912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25ae4d8-e24f-4f4e-9ddb-171c6c705fbb service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:21.926346 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 758/1521] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:19:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:19:26.689242 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05f87e-361a-47d1-a2fd-29dea15173eb None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:26.689818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05f87e-361a-47d1-a2fd-29dea15173eb None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:26.689818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05f87e-361a-47d1-a2fd-29dea15173eb None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:26.691456 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 14:19:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:26.696466 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745bf03-8ebb-4725-bbef-d5a49be86cee None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:26.696891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745bf03-8ebb-4725-bbef-d5a49be86cee None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:26.697028 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745bf03-8ebb-4725-bbef-d5a49be86cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:26.715361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b745bf03-8ebb-4725-bbef-d5a49be86cee None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:26.716583 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b745bf03-8ebb-4725-bbef-d5a49be86cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:26.726013 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 14:19:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:19:26.882010 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f05f87e-361a-47d1-a2fd-29dea15173eb None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:26.896127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f05f87e-361a-47d1-a2fd-29dea15173eb None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:26.901434 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70845dba-ed8f-4d41-bb60-6fe299cbccd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:26.902072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70845dba-ed8f-4d41-bb60-6fe299cbccd5 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:26.902374 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70845dba-ed8f-4d41-bb60-6fe299cbccd5 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:26.902634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70845dba-ed8f-4d41-bb60-6fe299cbccd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:26.910776 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70845dba-ed8f-4d41-bb60-6fe299cbccd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:26.912394 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70845dba-ed8f-4d41-bb60-6fe299cbccd5 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:26.922247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 765/1524] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:19:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:19:27.813324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6b0cd-a0ca-469e-bb42-ed4a0b7b9bd7 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:27.813892 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6b0cd-a0ca-469e-bb42-ed4a0b7b9bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:27.813892 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6b0cd-a0ca-469e-bb42-ed4a0b7b9bd7 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:27.815309 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 14:19:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:27.820180 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190fcf1-abaf-49ea-968b-afe67e995c50 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:27.820586 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190fcf1-abaf-49ea-968b-afe67e995c50 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:27.820742 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190fcf1-abaf-49ea-968b-afe67e995c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:27.839289 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-3190fcf1-abaf-49ea-968b-afe67e995c50 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:27.840480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3190fcf1-abaf-49ea-968b-afe67e995c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:27.850441 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 14:19:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:19:28.002206 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6b0cd-a0ca-469e-bb42-ed4a0b7b9bd7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:28.016334 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6b0cd-a0ca-469e-bb42-ed4a0b7b9bd7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:28.021546 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a0e98-84a3-47f8-ae70-4baf141c5d20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:28.022180 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a0e98-84a3-47f8-ae70-4baf141c5d20 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:28.022464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a0e98-84a3-47f8-ae70-4baf141c5d20 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:28.022739 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a0e98-84a3-47f8-ae70-4baf141c5d20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:28.031859 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a0e98-84a3-47f8-ae70-4baf141c5d20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:28.033455 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a0e98-84a3-47f8-ae70-4baf141c5d20 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:28.042671 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 761/1527] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:19:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:19:32.137453 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf514aa-fe42-4faf-b35c-a1d7c613921d None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:32.138566 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf514aa-fe42-4faf-b35c-a1d7c613921d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:32.138680 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf514aa-fe42-4faf-b35c-a1d7c613921d None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:32.140103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:19:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:32.145400 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28324b05-c2d8-402f-abaf-f347604a7e91 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:32.145851 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28324b05-c2d8-402f-abaf-f347604a7e91 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:32.145957 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28324b05-c2d8-402f-abaf-f347604a7e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:32.164972 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-28324b05-c2d8-402f-abaf-f347604a7e91 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:32.166248 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-28324b05-c2d8-402f-abaf-f347604a7e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:32.174663 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:19:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:19:32.212237 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf514aa-fe42-4faf-b35c-a1d7c613921d None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:32.213943 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7bf514aa-fe42-4faf-b35c-a1d7c613921d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:32.273087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf514aa-fe42-4faf-b35c-a1d7c613921d None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:32.276525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ed232-18d2-4cd4-9d36-1f32c6fc6251 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:32.277163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ed232-18d2-4cd4-9d36-1f32c6fc6251 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:32.277441 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ed232-18d2-4cd4-9d36-1f32c6fc6251 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:32.277713 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ed232-18d2-4cd4-9d36-1f32c6fc6251 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:32.287378 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ed232-18d2-4cd4-9d36-1f32c6fc6251 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:32.288846 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ed232-18d2-4cd4-9d36-1f32c6fc6251 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:32.294289 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:32.299845 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 768/1530] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:19:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:19:34.765508 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b66ef-dc34-45ff-bd7d-26fdf08b6efb None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:34.766050 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b66ef-dc34-45ff-bd7d-26fdf08b6efb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:34.766050 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b66ef-dc34-45ff-bd7d-26fdf08b6efb None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:34.767377 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:34.772487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ee410-0ac5-406a-9a4d-f3eabc35f249 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:34.772883 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ee410-0ac5-406a-9a4d-f3eabc35f249 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:34.773038 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ee410-0ac5-406a-9a4d-f3eabc35f249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:34.790763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-9d5ee410-0ac5-406a-9a4d-f3eabc35f249 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:34.791959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9d5ee410-0ac5-406a-9a4d-f3eabc35f249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:34.799935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:19:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:19:34.836189 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b66ef-dc34-45ff-bd7d-26fdf08b6efb None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:34.846886 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b66ef-dc34-45ff-bd7d-26fdf08b6efb None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:34.850563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f4cca-684a-46a7-a8a9-2469830ce853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:34.851211 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f4cca-684a-46a7-a8a9-2469830ce853 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:34.851526 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f4cca-684a-46a7-a8a9-2469830ce853 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:34.851806 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f4cca-684a-46a7-a8a9-2469830ce853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:34.861080 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f4cca-684a-46a7-a8a9-2469830ce853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:34.862513 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f4cca-684a-46a7-a8a9-2469830ce853 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:34.872356 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 764/1533] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:19:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:19:38.494995 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c600abb-c6dc-47e7-828a-0062caee20a7 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:38.495455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c600abb-c6dc-47e7-828a-0062caee20a7 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:38.495603 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c600abb-c6dc-47e7-828a-0062caee20a7 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:38.497557 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:19:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:38.503254 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff679606-7208-47f1-b536-545295a29126 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:38.503704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff679606-7208-47f1-b536-545295a29126 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:38.503826 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff679606-7208-47f1-b536-545295a29126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:38.524301 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ff679606-7208-47f1-b536-545295a29126 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:38.525503 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ff679606-7208-47f1-b536-545295a29126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:38.533430 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:19:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:19:38.805810 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c600abb-c6dc-47e7-828a-0062caee20a7 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:38.807497 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3c600abb-c6dc-47e7-828a-0062caee20a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:38.864258 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c600abb-c6dc-47e7-828a-0062caee20a7 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:38.867740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0660d275-8bdc-4878-ab48-8f47b5901249 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:38.868383 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660d275-8bdc-4878-ab48-8f47b5901249 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:38.868664 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660d275-8bdc-4878-ab48-8f47b5901249 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:38.868938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660d275-8bdc-4878-ab48-8f47b5901249 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:38.878441 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0660d275-8bdc-4878-ab48-8f47b5901249 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:38.879887 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0660d275-8bdc-4878-ab48-8f47b5901249 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:38.889344 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 771/1536] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:19:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:19:41.178810 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398784ad-5f73-4544-a0af-16302c55bf10 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:41.179441 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398784ad-5f73-4544-a0af-16302c55bf10 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:41.179441 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398784ad-5f73-4544-a0af-16302c55bf10 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:41.180756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:41.186817 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75471c4-a31b-4350-a262-f39eeeb0f14a None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:41.187286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75471c4-a31b-4350-a262-f39eeeb0f14a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:41.187390 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75471c4-a31b-4350-a262-f39eeeb0f14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:41.205465 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-b75471c4-a31b-4350-a262-f39eeeb0f14a None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:41.206651 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b75471c4-a31b-4350-a262-f39eeeb0f14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:41.214397 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:19:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:19:41.489067 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398784ad-5f73-4544-a0af-16302c55bf10 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:41.490738 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-398784ad-5f73-4544-a0af-16302c55bf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:41.503626 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398784ad-5f73-4544-a0af-16302c55bf10 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:41.507103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc678839-ba3c-4f2a-ae45-670165fde55f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:41.507746 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc678839-ba3c-4f2a-ae45-670165fde55f service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:41.508027 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc678839-ba3c-4f2a-ae45-670165fde55f service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:41.508300 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc678839-ba3c-4f2a-ae45-670165fde55f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:41.517019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc678839-ba3c-4f2a-ae45-670165fde55f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:41.518419 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc678839-ba3c-4f2a-ae45-670165fde55f service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:41.528421 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 767/1539] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:19:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:19:51.200204 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=32 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:51.523637 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=70 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:15.309636 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04155453-64ae-4291-a349-4a2a09cfe694 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:15.310990 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04155453-64ae-4291-a349-4a2a09cfe694 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:15.311533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04155453-64ae-4291-a349-4a2a09cfe694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:15.340398 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-04155453-64ae-4291-a349-4a2a09cfe694 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:15.341667 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-04155453-64ae-4291-a349-4a2a09cfe694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:15.355302 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 773/1540] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:15.364971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5eee95-ba51-4fa0-a59e-26925b0390e6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:15.366059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5eee95-ba51-4fa0-a59e-26925b0390e6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:15.366679 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5eee95-ba51-4fa0-a59e-26925b0390e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:15.393731 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8e5eee95-ba51-4fa0-a59e-26925b0390e6 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:15.395512 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8e5eee95-ba51-4fa0-a59e-26925b0390e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:15.410159 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 768/1541] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:15.415490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04155453-64ae-4291-a349-4a2a09cfe694 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:15.428355 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04155453-64ae-4291-a349-4a2a09cfe694 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:15.447748 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12edd065-e518-4edb-9025-ed99eb558ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:15.448791 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edd065-e518-4edb-9025-ed99eb558ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:15.449089 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edd065-e518-4edb-9025-ed99eb558ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:15.449442 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edd065-e518-4edb-9025-ed99eb558ac5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:15.450340 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12edd065-e518-4edb-9025-ed99eb558ac5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:15.450677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12edd065-e518-4edb-9025-ed99eb558ac5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:15.456675 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12edd065-e518-4edb-9025-ed99eb558ac5 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:15.479042 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 774/1542] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:21:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:15.487229 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5eee95-ba51-4fa0-a59e-26925b0390e6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:15.504149 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5eee95-ba51-4fa0-a59e-26925b0390e6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:15.516157 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c2f4a-67cb-4a86-b8e9-21205f6f8569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:15.517163 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c2f4a-67cb-4a86-b8e9-21205f6f8569 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:15.517674 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c2f4a-67cb-4a86-b8e9-21205f6f8569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:15.518164 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c2f4a-67cb-4a86-b8e9-21205f6f8569 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:15.519270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c2f4a-67cb-4a86-b8e9-21205f6f8569 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:15.521623 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c2f4a-67cb-4a86-b8e9-21205f6f8569 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:15.564093 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 769/1543] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:15 2022] POST /identity/v3/projects => generated 403 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:15.564310 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398455b-1d6f-4c46-b0db-75c6d25b374e None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:15.564606 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398455b-1d6f-4c46-b0db-75c6d25b374e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:15.564888 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398455b-1d6f-4c46-b0db-75c6d25b374e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:15.569041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c2f4a-67cb-4a86-b8e9-21205f6f8569 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:15.604443 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-e398455b-1d6f-4c46-b0db-75c6d25b374e None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:15.605890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e398455b-1d6f-4c46-b0db-75c6d25b374e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:15.700690 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c2f4a-67cb-4a86-b8e9-21205f6f8569 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:15.715672 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3006fd-c560-4017-9a45-97a8a0e9281e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:15.716769 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3006fd-c560-4017-9a45-97a8a0e9281e admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:15.717285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3006fd-c560-4017-9a45-97a8a0e9281e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:15.717778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3006fd-c560-4017-9a45-97a8a0e9281e admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:15.720047 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3006fd-c560-4017-9a45-97a8a0e9281e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:15.721316 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-0c3006fd-c560-4017-9a45-97a8a0e9281e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:21:15.721316 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: warnings.warn(msg) Jun 07 14:21:15.721316 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: Jun 07 14:21:15.723073 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3006fd-c560-4017-9a45-97a8a0e9281e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:15.759905 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 770/1544] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:15 2022] POST /identity/v3/users => generated 302 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:21:15.763021 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d62352-7529-45ca-a36c-8f4344796e31 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:15.763699 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d62352-7529-45ca-a36c-8f4344796e31 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:15.764432 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d62352-7529-45ca-a36c-8f4344796e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:15.767780 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 775/1545] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:21:15.770336 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73eef3-baf0-47fc-9cdd-72423eee7a8e None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:15.771159 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73eef3-baf0-47fc-9cdd-72423eee7a8e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:15.772263 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73eef3-baf0-47fc-9cdd-72423eee7a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:15.793797 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d9d62352-7529-45ca-a36c-8f4344796e31 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:15.795451 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d9d62352-7529-45ca-a36c-8f4344796e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:15.804362 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-4e73eef3-baf0-47fc-9cdd-72423eee7a8e None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:15.805557 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4e73eef3-baf0-47fc-9cdd-72423eee7a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:15.806155 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 771/1546] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:15.807443 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d09ebe-27e8-49fb-806f-f75192d95ce1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:15.807726 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d09ebe-27e8-49fb-806f-f75192d95ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:15.807996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d09ebe-27e8-49fb-806f-f75192d95ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:15.819024 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 776/1547] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:15.820194 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73eef3-baf0-47fc-9cdd-72423eee7a8e None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:15.826419 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f3d09ebe-27e8-49fb-806f-f75192d95ce1 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:15.827681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f3d09ebe-27e8-49fb-806f-f75192d95ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:15.829621 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73eef3-baf0-47fc-9cdd-72423eee7a8e None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:15.834036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb27cc7-d908-4c77-8a55-e2abf096ad41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:15.834715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb27cc7-d908-4c77-8a55-e2abf096ad41 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:15.835002 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb27cc7-d908-4c77-8a55-e2abf096ad41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:15.835302 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 772/1548] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:15.835458 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb27cc7-d908-4c77-8a55-e2abf096ad41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:15.836035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb27cc7-d908-4c77-8a55-e2abf096ad41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:15.836504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92bc50-53c8-496f-9cd8-cb99de3dc0bf None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:15.836786 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92bc50-53c8-496f-9cd8-cb99de3dc0bf None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:15.837059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92bc50-53c8-496f-9cd8-cb99de3dc0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:15.837646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb27cc7-d908-4c77-8a55-e2abf096ad41 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:15.849873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 777/1549] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:15.851439 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8c181-64aa-4f62-b0f4-d20148e072b7 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:15.852179 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8c181-64aa-4f62-b0f4-d20148e072b7 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:15.852645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8c181-64aa-4f62-b0f4-d20148e072b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:15.855164 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5a92bc50-53c8-496f-9cd8-cb99de3dc0bf None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:15.856345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5a92bc50-53c8-496f-9cd8-cb99de3dc0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:15.864630 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 773/1550] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:15.865860 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b922c4b-57af-4685-8fc3-979f6c58893c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:15.866146 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b922c4b-57af-4685-8fc3-979f6c58893c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:15.866421 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b922c4b-57af-4685-8fc3-979f6c58893c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:15.871893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-7ca8c181-64aa-4f62-b0f4-d20148e072b7 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:15.873062 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7ca8c181-64aa-4f62-b0f4-d20148e072b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:15.881332 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 778/1551] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:15.882370 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc62c50-1b63-415e-87e9-56aa1580e14d None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:15.882652 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc62c50-1b63-415e-87e9-56aa1580e14d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:15.882926 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc62c50-1b63-415e-87e9-56aa1580e14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:15.883729 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4b922c4b-57af-4685-8fc3-979f6c58893c None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:15.884898 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4b922c4b-57af-4685-8fc3-979f6c58893c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:15.892619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 774/1552] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:15.893854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96968718-afa8-4b8d-825c-aac435031b02 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:15.894139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96968718-afa8-4b8d-825c-aac435031b02 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:15.894412 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96968718-afa8-4b8d-825c-aac435031b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:15.901136 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-7fc62c50-1b63-415e-87e9-56aa1580e14d None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:15.902366 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7fc62c50-1b63-415e-87e9-56aa1580e14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:15.911079 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 779/1553] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:15.912447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91330d-29fc-496e-9d2f-2261cc58981f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:15.912858 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91330d-29fc-496e-9d2f-2261cc58981f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:15.913152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91330d-29fc-496e-9d2f-2261cc58981f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:15.913289 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-96968718-afa8-4b8d-825c-aac435031b02 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:15.914550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-96968718-afa8-4b8d-825c-aac435031b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:15.923440 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 775/1554] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:15.924776 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffe731-01fb-41d9-a113-ed922807d74d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:15.925067 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffe731-01fb-41d9-a113-ed922807d74d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:15.925347 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffe731-01fb-41d9-a113-ed922807d74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:15.932610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-4a91330d-29fc-496e-9d2f-2261cc58981f None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:15.933790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4a91330d-29fc-496e-9d2f-2261cc58981f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:15.941633 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 780/1555] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:15.942536 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a91330d-29fc-496e-9d2f-2261cc58981f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:15.942932 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b1ffe731-01fb-41d9-a113-ed922807d74d None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:15.944075 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b1ffe731-01fb-41d9-a113-ed922807d74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:15.951947 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a91330d-29fc-496e-9d2f-2261cc58981f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:15.952087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 776/1556] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:15.952853 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffe731-01fb-41d9-a113-ed922807d74d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:15.958603 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b9c939-5865-4dfa-b930-a452fa4ddc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:15.959978 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9c939-5865-4dfa-b930-a452fa4ddc3b admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:15.960467 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9c939-5865-4dfa-b930-a452fa4ddc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:15.960792 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9c939-5865-4dfa-b930-a452fa4ddc3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:15.961664 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b9c939-5865-4dfa-b930-a452fa4ddc3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:15.962794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffe731-01fb-41d9-a113-ed922807d74d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:15.963494 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b9c939-5865-4dfa-b930-a452fa4ddc3b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:15.969755 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6043f13-9298-43c2-837b-8a4e43016c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:15.970879 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6043f13-9298-43c2-837b-8a4e43016c40 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:15.971165 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6043f13-9298-43c2-837b-8a4e43016c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:15.971536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6043f13-9298-43c2-837b-8a4e43016c40 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:15.972264 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6043f13-9298-43c2-837b-8a4e43016c40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:15.972566 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6043f13-9298-43c2-837b-8a4e43016c40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:15.973562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-e6043f13-9298-43c2-837b-8a4e43016c40 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:21:15.973562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: warnings.warn(msg) Jun 07 14:21:15.973562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: Jun 07 14:21:15.974446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6043f13-9298-43c2-837b-8a4e43016c40 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:15.974972 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 781/1557] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:15.976438 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e22a0-a797-4fa4-a2de-907e6882a084 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:15.976986 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e22a0-a797-4fa4-a2de-907e6882a084 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:15.977291 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e22a0-a797-4fa4-a2de-907e6882a084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:15.991265 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 777/1558] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:21:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:15.991265 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d2601-d8ab-4995-a67d-d3ec505e5139 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:15.991555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d2601-d8ab-4995-a67d-d3ec505e5139 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:15.991779 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d2601-d8ab-4995-a67d-d3ec505e5139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:15.996074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-e39e22a0-a797-4fa4-a2de-907e6882a084 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:15.997612 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e39e22a0-a797-4fa4-a2de-907e6882a084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:16.005940 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 782/1559] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:16.007077 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b498a62-5a75-467a-9e84-2852ad2cec13 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.007410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b498a62-5a75-467a-9e84-2852ad2cec13 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.007721 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b498a62-5a75-467a-9e84-2852ad2cec13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.013287 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c65d2601-d8ab-4995-a67d-d3ec505e5139 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:16.014464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c65d2601-d8ab-4995-a67d-d3ec505e5139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:16.022618 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 778/1560] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:16.026431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d2601-d8ab-4995-a67d-d3ec505e5139 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.026535 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-5b498a62-5a75-467a-9e84-2852ad2cec13 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:16.027424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5b498a62-5a75-467a-9e84-2852ad2cec13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:16.033732 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d2601-d8ab-4995-a67d-d3ec505e5139 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.036302 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 783/1561] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:16.037068 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b498a62-5a75-467a-9e84-2852ad2cec13 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.038622 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908d0fb-4c93-4ed3-896a-c1d3d97ad566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.039563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908d0fb-4c93-4ed3-896a-c1d3d97ad566 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.039884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908d0fb-4c93-4ed3-896a-c1d3d97ad566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.040166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908d0fb-4c93-4ed3-896a-c1d3d97ad566 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.041037 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908d0fb-4c93-4ed3-896a-c1d3d97ad566 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:16.041456 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908d0fb-4c93-4ed3-896a-c1d3d97ad566 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.042983 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908d0fb-4c93-4ed3-896a-c1d3d97ad566 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.046069 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b498a62-5a75-467a-9e84-2852ad2cec13 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.051104 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17652473-33ae-4335-9645-68d3404759a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.051766 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17652473-33ae-4335-9645-68d3404759a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.052044 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17652473-33ae-4335-9645-68d3404759a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.052320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17652473-33ae-4335-9645-68d3404759a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.053035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17652473-33ae-4335-9645-68d3404759a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:16.053335 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17652473-33ae-4335-9645-68d3404759a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.054939 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17652473-33ae-4335-9645-68d3404759a1 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.062753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 779/1562] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:16.062753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908d0fb-4c93-4ed3-896a-c1d3d97ad566 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.073302 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 784/1563] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:16.073302 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17652473-33ae-4335-9645-68d3404759a1 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.073562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908d0fb-4c93-4ed3-896a-c1d3d97ad566 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.073562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d7e50-37b9-4ce6-8116-1ebe869eac86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.073562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d7e50-37b9-4ce6-8116-1ebe869eac86 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.073869 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d7e50-37b9-4ce6-8116-1ebe869eac86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.073959 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d7e50-37b9-4ce6-8116-1ebe869eac86 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.074640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d7e50-37b9-4ce6-8116-1ebe869eac86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:16.074947 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d7e50-37b9-4ce6-8116-1ebe869eac86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.076574 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d7e50-37b9-4ce6-8116-1ebe869eac86 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.077117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17652473-33ae-4335-9645-68d3404759a1 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.082127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60a9fc-4a54-4a26-aa6a-00654eb5a49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.082991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60a9fc-4a54-4a26-aa6a-00654eb5a49c admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.083306 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60a9fc-4a54-4a26-aa6a-00654eb5a49c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.083581 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60a9fc-4a54-4a26-aa6a-00654eb5a49c admin admin] PATH_INFO: `/v3/projects/bcc308a8cf1f4e75ba30dfd1c84dbbfe/users/0cb00f50054442749dc681eb800e8117/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.095857 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 780/1564] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:16.095857 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d7e50-37b9-4ce6-8116-1ebe869eac86 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.100563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d7e50-37b9-4ce6-8116-1ebe869eac86 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.102471 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60a9fc-4a54-4a26-aa6a-00654eb5a49c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcc308a8cf1f4e75ba30dfd1c84dbbfe, user_id=0cb00f50054442749dc681eb800e8117, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.103527 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: WARNING py.warnings [None req-5c60a9fc-4a54-4a26-aa6a-00654eb5a49c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:21:16.103527 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: warnings.warn(msg) Jun 07 14:21:16.103527 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: Jun 07 14:21:16.105378 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213ce78-2a60-4488-9d2d-17e679f7bbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.105752 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60a9fc-4a54-4a26-aa6a-00654eb5a49c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.106019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213ce78-2a60-4488-9d2d-17e679f7bbd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.106307 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213ce78-2a60-4488-9d2d-17e679f7bbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.106586 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213ce78-2a60-4488-9d2d-17e679f7bbd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.107300 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213ce78-2a60-4488-9d2d-17e679f7bbd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.109242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213ce78-2a60-4488-9d2d-17e679f7bbd5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.119805 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 785/1565] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:16 2022] PUT /identity/v3/projects/bcc308a8cf1f4e75ba30dfd1c84dbbfe/users/0cb00f50054442749dc681eb800e8117/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:16.120830 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60a9fc-4a54-4a26-aa6a-00654eb5a49c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.135889 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 781/1566] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:16 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:16.136925 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213ce78-2a60-4488-9d2d-17e679f7bbd5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.153167 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:16.173770 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:16.191433 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60a9fc-4a54-4a26-aa6a-00654eb5a49c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.204707 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213ce78-2a60-4488-9d2d-17e679f7bbd5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.209103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191a035-b025-45b8-aee6-3f99bf9de063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.209774 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191a035-b025-45b8-aee6-3f99bf9de063 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.210052 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191a035-b025-45b8-aee6-3f99bf9de063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.210334 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191a035-b025-45b8-aee6-3f99bf9de063 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.211072 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191a035-b025-45b8-aee6-3f99bf9de063 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:16.211415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191a035-b025-45b8-aee6-3f99bf9de063 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.213010 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191a035-b025-45b8-aee6-3f99bf9de063 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.225821 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 782/1567] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:16.227129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191a035-b025-45b8-aee6-3f99bf9de063 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.240068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191a035-b025-45b8-aee6-3f99bf9de063 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.245302 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8e90b-5c1d-4df4-850c-465aceb272c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.245961 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8e90b-5c1d-4df4-850c-465aceb272c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.246246 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8e90b-5c1d-4df4-850c-465aceb272c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.246523 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8e90b-5c1d-4df4-850c-465aceb272c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.247280 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8e90b-5c1d-4df4-850c-465aceb272c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:16.247595 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8e90b-5c1d-4df4-850c-465aceb272c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.249182 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8e90b-5c1d-4df4-850c-465aceb272c5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.258286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd0cf8-525a-4f93-8066-1c12dd619aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.258957 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd0cf8-525a-4f93-8066-1c12dd619aae admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.259300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd0cf8-525a-4f93-8066-1c12dd619aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.259539 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd0cf8-525a-4f93-8066-1c12dd619aae admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.260265 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd0cf8-525a-4f93-8066-1c12dd619aae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:16.260570 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd0cf8-525a-4f93-8066-1c12dd619aae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.269832 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 783/1568] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:16.269984 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd0cf8-525a-4f93-8066-1c12dd619aae admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.270086 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8e90b-5c1d-4df4-850c-465aceb272c5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.273889 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8e90b-5c1d-4df4-850c-465aceb272c5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.278478 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c6836-640d-4bfb-8802-45e28d562553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.279115 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c6836-640d-4bfb-8802-45e28d562553 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.279426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c6836-640d-4bfb-8802-45e28d562553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.279700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c6836-640d-4bfb-8802-45e28d562553 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.280371 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c6836-640d-4bfb-8802-45e28d562553 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.280683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 786/1569] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:16.282478 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd0cf8-525a-4f93-8066-1c12dd619aae admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.283079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c6836-640d-4bfb-8802-45e28d562553 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.297837 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd0cf8-525a-4f93-8066-1c12dd619aae admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.302592 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95450ec-6c0a-4ba0-9d61-09782904b341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.303270 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95450ec-6c0a-4ba0-9d61-09782904b341 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.303557 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95450ec-6c0a-4ba0-9d61-09782904b341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.303836 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95450ec-6c0a-4ba0-9d61-09782904b341 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.304560 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95450ec-6c0a-4ba0-9d61-09782904b341 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.305528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: WARNING py.warnings [None req-f95450ec-6c0a-4ba0-9d61-09782904b341 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:21:16.305528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: warnings.warn(msg) Jun 07 14:21:16.305528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: Jun 07 14:21:16.306708 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95450ec-6c0a-4ba0-9d61-09782904b341 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.311318 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 784/1570] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:16 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:16.312554 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c6836-640d-4bfb-8802-45e28d562553 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.333381 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 787/1571] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:16 2022] POST /identity/v3/projects => generated 391 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:16.335426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad88395-08da-4e2c-82f8-df9c2189e80c None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.335711 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad88395-08da-4e2c-82f8-df9c2189e80c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.335983 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad88395-08da-4e2c-82f8-df9c2189e80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.353743 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-aad88395-08da-4e2c-82f8-df9c2189e80c None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:16.354921 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-aad88395-08da-4e2c-82f8-df9c2189e80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:16.377550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c6836-640d-4bfb-8802-45e28d562553 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.414104 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 788/1572] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:16.417353 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad88395-08da-4e2c-82f8-df9c2189e80c None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.424889 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad88395-08da-4e2c-82f8-df9c2189e80c None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.429396 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e84f63-e81e-48af-bd94-4719c329df63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.430057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e84f63-e81e-48af-bd94-4719c329df63 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.430332 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e84f63-e81e-48af-bd94-4719c329df63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.430603 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e84f63-e81e-48af-bd94-4719c329df63 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.431310 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e84f63-e81e-48af-bd94-4719c329df63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.433216 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e84f63-e81e-48af-bd94-4719c329df63 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.434339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae87d3-42a1-4e58-ad9d-eda296eebe3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.435006 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae87d3-42a1-4e58-ad9d-eda296eebe3b admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.435333 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae87d3-42a1-4e58-ad9d-eda296eebe3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.435612 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae87d3-42a1-4e58-ad9d-eda296eebe3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.436303 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ae87d3-42a1-4e58-ad9d-eda296eebe3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.439132 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ae87d3-42a1-4e58-ad9d-eda296eebe3b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.453352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 789/1573] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:16 2022] POST /identity/v3/users => generated 293 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:16.454249 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e84f63-e81e-48af-bd94-4719c329df63 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.464089 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e84f63-e81e-48af-bd94-4719c329df63 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.466497 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 785/1574] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:16 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:16.467412 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae87d3-42a1-4e58-ad9d-eda296eebe3b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.469444 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1478724-7821-4cdc-9ff5-bd198f1fcbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.470121 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1478724-7821-4cdc-9ff5-bd198f1fcbbe admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.470646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1478724-7821-4cdc-9ff5-bd198f1fcbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.470937 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1478724-7821-4cdc-9ff5-bd198f1fcbbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.473981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1478724-7821-4cdc-9ff5-bd198f1fcbbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.473981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1478724-7821-4cdc-9ff5-bd198f1fcbbe admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.475170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:16.501997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 790/1575] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:16 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:16.503067 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1478724-7821-4cdc-9ff5-bd198f1fcbbe admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.530349 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:16.539142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae87d3-42a1-4e58-ad9d-eda296eebe3b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.568353 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1478724-7821-4cdc-9ff5-bd198f1fcbbe admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.573435 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21e84d-154a-4f7b-acc7-47b8235cf289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.574097 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21e84d-154a-4f7b-acc7-47b8235cf289 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.574381 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21e84d-154a-4f7b-acc7-47b8235cf289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.574658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21e84d-154a-4f7b-acc7-47b8235cf289 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.575497 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21e84d-154a-4f7b-acc7-47b8235cf289 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.577528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21e84d-154a-4f7b-acc7-47b8235cf289 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.596076 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84fe125-2837-450f-b16d-928bb3650cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.596860 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fe125-2837-450f-b16d-928bb3650cea admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.597265 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fe125-2837-450f-b16d-928bb3650cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.597544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fe125-2837-450f-b16d-928bb3650cea admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.598284 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84fe125-2837-450f-b16d-928bb3650cea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.600220 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84fe125-2837-450f-b16d-928bb3650cea admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.604456 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 791/1576] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:16 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:16.605440 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21e84d-154a-4f7b-acc7-47b8235cf289 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.629149 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 786/1577] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:16 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:16.629967 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84fe125-2837-450f-b16d-928bb3650cea admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.675049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21e84d-154a-4f7b-acc7-47b8235cf289 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.691809 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84fe125-2837-450f-b16d-928bb3650cea admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.696364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa05dc-a818-4380-934c-9285d05f4930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.697036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa05dc-a818-4380-934c-9285d05f4930 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.697335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa05dc-a818-4380-934c-9285d05f4930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.697599 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa05dc-a818-4380-934c-9285d05f4930 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.698315 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa05dc-a818-4380-934c-9285d05f4930 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.700317 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa05dc-a818-4380-934c-9285d05f4930 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.720878 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 787/1578] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:16 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:16.721621 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa05dc-a818-4380-934c-9285d05f4930 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.732079 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d94c89-7725-4590-aa01-d62f51a0fbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.732767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d94c89-7725-4590-aa01-d62f51a0fbec admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.733080 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d94c89-7725-4590-aa01-d62f51a0fbec admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.733359 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d94c89-7725-4590-aa01-d62f51a0fbec admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.734801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d94c89-7725-4590-aa01-d62f51a0fbec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.736807 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d94c89-7725-4590-aa01-d62f51a0fbec admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.753019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:16.758339 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 792/1579] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:16 2022] POST /identity/v3/users => generated 285 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:16.758997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d94c89-7725-4590-aa01-d62f51a0fbec admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.768997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d94c89-7725-4590-aa01-d62f51a0fbec admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.773678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d33d55-1963-4f63-a2e2-90c729007ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.774328 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d33d55-1963-4f63-a2e2-90c729007ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.774638 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d33d55-1963-4f63-a2e2-90c729007ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.774915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d33d55-1963-4f63-a2e2-90c729007ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.775668 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d33d55-1963-4f63-a2e2-90c729007ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.777318 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d33d55-1963-4f63-a2e2-90c729007ad5 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.779984 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa05dc-a818-4380-934c-9285d05f4930 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.784393 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba74a4f-c665-464f-9559-11ebf61a0125 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.785047 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba74a4f-c665-464f-9559-11ebf61a0125 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.785333 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba74a4f-c665-464f-9559-11ebf61a0125 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.785609 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba74a4f-c665-464f-9559-11ebf61a0125 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.790869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 793/1580] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:16.790869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d33d55-1963-4f63-a2e2-90c729007ad5 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.798323 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d33d55-1963-4f63-a2e2-90c729007ad5 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.798676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba74a4f-c665-464f-9559-11ebf61a0125 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.800203 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba74a4f-c665-464f-9559-11ebf61a0125 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.802888 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f69fc-a33a-4cc0-a9e4-c0447d09c4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.803553 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f69fc-a33a-4cc0-a9e4-c0447d09c4d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.803832 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f69fc-a33a-4cc0-a9e4-c0447d09c4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.804106 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f69fc-a33a-4cc0-a9e4-c0447d09c4d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.804772 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f69fc-a33a-4cc0-a9e4-c0447d09c4d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.806649 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f69fc-a33a-4cc0-a9e4-c0447d09c4d7 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.810089 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 788/1581] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:16.810700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba74a4f-c665-464f-9559-11ebf61a0125 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.820131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba74a4f-c665-464f-9559-11ebf61a0125 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.825024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd878f60-f48c-4a89-974f-632716759ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.825717 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd878f60-f48c-4a89-974f-632716759ec1 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.826002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd878f60-f48c-4a89-974f-632716759ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.826278 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd878f60-f48c-4a89-974f-632716759ec1 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.826970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd878f60-f48c-4a89-974f-632716759ec1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.828062 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 794/1582] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:16 2022] POST /identity/v3/users => generated 290 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:16.828635 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f69fc-a33a-4cc0-a9e4-c0447d09c4d7 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.832659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd878f60-f48c-4a89-974f-632716759ec1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.838308 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f69fc-a33a-4cc0-a9e4-c0447d09c4d7 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.844813 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954fed06-9e56-46a7-a5d8-fe16e7b92d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.845612 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fed06-9e56-46a7-a5d8-fe16e7b92d48 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.846010 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fed06-9e56-46a7-a5d8-fe16e7b92d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.846295 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fed06-9e56-46a7-a5d8-fe16e7b92d48 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.847505 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954fed06-9e56-46a7-a5d8-fe16e7b92d48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.849508 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954fed06-9e56-46a7-a5d8-fe16e7b92d48 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.860580 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 789/1583] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:16 2022] POST /identity/v3/users => generated 289 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:16.860580 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd878f60-f48c-4a89-974f-632716759ec1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.873287 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:16.878228 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd878f60-f48c-4a89-974f-632716759ec1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.884837 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 795/1584] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:16 2022] POST /identity/v3/users => generated 288 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:16.885632 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954fed06-9e56-46a7-a5d8-fe16e7b92d48 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.887004 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e14f6-d635-4a6c-a913-7c4c7f4b5716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.887826 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e14f6-d635-4a6c-a913-7c4c7f4b5716 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.888207 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e14f6-d635-4a6c-a913-7c4c7f4b5716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.888578 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e14f6-d635-4a6c-a913-7c4c7f4b5716 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.889370 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e14f6-d635-4a6c-a913-7c4c7f4b5716 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.892014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e14f6-d635-4a6c-a913-7c4c7f4b5716 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.896979 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954fed06-9e56-46a7-a5d8-fe16e7b92d48 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.902008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88102e22-03b7-43b6-bfe8-c6babc951ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.902921 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88102e22-03b7-43b6-bfe8-c6babc951ade admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.903047 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88102e22-03b7-43b6-bfe8-c6babc951ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.903338 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88102e22-03b7-43b6-bfe8-c6babc951ade admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.904331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88102e22-03b7-43b6-bfe8-c6babc951ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.906341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88102e22-03b7-43b6-bfe8-c6babc951ade admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.916612 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 790/1585] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:16 2022] POST /identity/v3/users => generated 293 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:16.921850 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 796/1586] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:16.921850 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88102e22-03b7-43b6-bfe8-c6babc951ade admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.922118 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e14f6-d635-4a6c-a913-7c4c7f4b5716 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.932457 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88102e22-03b7-43b6-bfe8-c6babc951ade admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.934031 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e14f6-d635-4a6c-a913-7c4c7f4b5716 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.937369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0d0b1-006e-49b1-a4c7-d0e00e23bb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.938118 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0d0b1-006e-49b1-a4c7-d0e00e23bb1c admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.938476 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0d0b1-006e-49b1-a4c7-d0e00e23bb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.938821 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0d0b1-006e-49b1-a4c7-d0e00e23bb1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.939575 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abefe824-a80f-4a40-b962-587697da3648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.939697 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0d0b1-006e-49b1-a4c7-d0e00e23bb1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.940245 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abefe824-a80f-4a40-b962-587697da3648 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.940532 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abefe824-a80f-4a40-b962-587697da3648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.940813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abefe824-a80f-4a40-b962-587697da3648 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.941541 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0d0b1-006e-49b1-a4c7-d0e00e23bb1c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.941661 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abefe824-a80f-4a40-b962-587697da3648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.943304 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abefe824-a80f-4a40-b962-587697da3648 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.954074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 797/1587] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:16.955019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0d0b1-006e-49b1-a4c7-d0e00e23bb1c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.955748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 791/1588] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:16.956663 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abefe824-a80f-4a40-b962-587697da3648 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.967670 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abefe824-a80f-4a40-b962-587697da3648 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.968793 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0d0b1-006e-49b1-a4c7-d0e00e23bb1c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.973222 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53fd17-0a9c-44e4-ad53-4b13c623dc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.973452 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b72fdf-12bf-421e-8a5c-f9a9929f8ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.973861 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53fd17-0a9c-44e4-ad53-4b13c623dc41 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.974133 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b72fdf-12bf-421e-8a5c-f9a9929f8ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.974247 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53fd17-0a9c-44e4-ad53-4b13c623dc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.974404 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b72fdf-12bf-421e-8a5c-f9a9929f8ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.974519 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53fd17-0a9c-44e4-ad53-4b13c623dc41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.974683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b72fdf-12bf-421e-8a5c-f9a9929f8ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.975149 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53fd17-0a9c-44e4-ad53-4b13c623dc41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.975450 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b72fdf-12bf-421e-8a5c-f9a9929f8ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.977050 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53fd17-0a9c-44e4-ad53-4b13c623dc41 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.977290 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b72fdf-12bf-421e-8a5c-f9a9929f8ae4 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.988167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 798/1589] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:16.991380 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b72fdf-12bf-421e-8a5c-f9a9929f8ae4 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.991525 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 792/1590] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:16.991525 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53fd17-0a9c-44e4-ad53-4b13c623dc41 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:17.000703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53fd17-0a9c-44e4-ad53-4b13c623dc41 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:17.002922 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b72fdf-12bf-421e-8a5c-f9a9929f8ae4 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:17.005915 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5201d-4951-435c-a22a-501823e5248f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:17.006590 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5201d-4951-435c-a22a-501823e5248f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.006875 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5201d-4951-435c-a22a-501823e5248f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.007155 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5201d-4951-435c-a22a-501823e5248f admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.007939 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5201d-4951-435c-a22a-501823e5248f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:17.008268 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f59c51-9405-44c2-8791-c78fa6aa1ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:17.009142 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f59c51-9405-44c2-8791-c78fa6aa1ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.009272 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f59c51-9405-44c2-8791-c78fa6aa1ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.009487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f59c51-9405-44c2-8791-c78fa6aa1ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.009594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5201d-4951-435c-a22a-501823e5248f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:17.010233 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f59c51-9405-44c2-8791-c78fa6aa1ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:17.012176 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f59c51-9405-44c2-8791-c78fa6aa1ce2 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:17.020090 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 793/1591] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:17.020931 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5201d-4951-435c-a22a-501823e5248f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:17.026508 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 799/1592] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:17.027457 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f59c51-9405-44c2-8791-c78fa6aa1ce2 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:17.040280 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5201d-4951-435c-a22a-501823e5248f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:17.045286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f59c51-9405-44c2-8791-c78fa6aa1ce2 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:17.047977 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0272d-53d5-4a7a-99a4-7d62c1f6abfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:17.048767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0272d-53d5-4a7a-99a4-7d62c1f6abfa admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.049145 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0272d-53d5-4a7a-99a4-7d62c1f6abfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.049798 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0272d-53d5-4a7a-99a4-7d62c1f6abfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.050557 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0272d-53d5-4a7a-99a4-7d62c1f6abfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:17.051615 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ead69f-c3c7-433a-90d3-8fe51d8bdf21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:17.052459 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0272d-53d5-4a7a-99a4-7d62c1f6abfa admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:17.053803 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ead69f-c3c7-433a-90d3-8fe51d8bdf21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.053803 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ead69f-c3c7-433a-90d3-8fe51d8bdf21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.053803 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ead69f-c3c7-433a-90d3-8fe51d8bdf21 admin admin] PATH_INFO: `/v3/projects/a7844e15ebe643e1b176d40b6cbf3ace/users/f6efd5bf855c47fe8ea69f4944bd2d0f/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.065920 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 794/1593] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:17.066893 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0272d-53d5-4a7a-99a4-7d62c1f6abfa admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:17.071673 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ead69f-c3c7-433a-90d3-8fe51d8bdf21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7844e15ebe643e1b176d40b6cbf3ace, user_id=f6efd5bf855c47fe8ea69f4944bd2d0f, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:17.074457 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ead69f-c3c7-433a-90d3-8fe51d8bdf21 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:17.076896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0272d-53d5-4a7a-99a4-7d62c1f6abfa admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:17.082652 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6569b81-1520-4627-85d8-350e35283ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:17.083506 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6569b81-1520-4627-85d8-350e35283ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.083885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6569b81-1520-4627-85d8-350e35283ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.084235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6569b81-1520-4627-85d8-350e35283ed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.085045 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6569b81-1520-4627-85d8-350e35283ed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:17.086784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6569b81-1520-4627-85d8-350e35283ed5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:17.089547 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 800/1594] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:17 2022] PUT /identity/v3/projects/a7844e15ebe643e1b176d40b6cbf3ace/users/f6efd5bf855c47fe8ea69f4944bd2d0f/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:17.096477 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ead69f-c3c7-433a-90d3-8fe51d8bdf21 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:17.098823 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 795/1595] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:17.099970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6569b81-1520-4627-85d8-350e35283ed5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:17.112847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6569b81-1520-4627-85d8-350e35283ed5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:17.170670 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ead69f-c3c7-433a-90d3-8fe51d8bdf21 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:17.177748 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19daf224-9a58-4eac-8a84-43780e48c7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:17.178536 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19daf224-9a58-4eac-8a84-43780e48c7fe admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.178822 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19daf224-9a58-4eac-8a84-43780e48c7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.179100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19daf224-9a58-4eac-8a84-43780e48c7fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.179907 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19daf224-9a58-4eac-8a84-43780e48c7fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:17.181515 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19daf224-9a58-4eac-8a84-43780e48c7fe admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:17.189823 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea9c97-c83a-4afb-82c1-d82d3a569d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:17.190579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea9c97-c83a-4afb-82c1-d82d3a569d6f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.190895 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea9c97-c83a-4afb-82c1-d82d3a569d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.191139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea9c97-c83a-4afb-82c1-d82d3a569d6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.191927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ea9c97-c83a-4afb-82c1-d82d3a569d6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:17.193686 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ea9c97-c83a-4afb-82c1-d82d3a569d6f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:17.197161 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 801/1596] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:17.197161 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19daf224-9a58-4eac-8a84-43780e48c7fe admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:17.216668 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19daf224-9a58-4eac-8a84-43780e48c7fe admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:17.222898 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df65e6-b278-40c6-84fb-a8a54633165f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:17.223663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df65e6-b278-40c6-84fb-a8a54633165f admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.224010 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df65e6-b278-40c6-84fb-a8a54633165f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.224188 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df65e6-b278-40c6-84fb-a8a54633165f admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.225024 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df65e6-b278-40c6-84fb-a8a54633165f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:17.227668 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df65e6-b278-40c6-84fb-a8a54633165f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:17.238698 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 796/1597] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:17.239869 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea9c97-c83a-4afb-82c1-d82d3a569d6f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:17.244761 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 802/1598] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:17.245687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df65e6-b278-40c6-84fb-a8a54633165f admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:17.253444 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea9c97-c83a-4afb-82c1-d82d3a569d6f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:17.256570 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df65e6-b278-40c6-84fb-a8a54633165f admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:17.258349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e147e1-47f8-4ed0-8c13-efde97540ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:17.259161 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e147e1-47f8-4ed0-8c13-efde97540ccf admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.259541 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e147e1-47f8-4ed0-8c13-efde97540ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.259888 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e147e1-47f8-4ed0-8c13-efde97540ccf admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.261163 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e147e1-47f8-4ed0-8c13-efde97540ccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:17.262324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7e2a4-8fb7-4735-9e4d-7ce3905950de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:17.263030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e147e1-47f8-4ed0-8c13-efde97540ccf admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:17.263236 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7e2a4-8fb7-4735-9e4d-7ce3905950de admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.263527 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7e2a4-8fb7-4735-9e4d-7ce3905950de admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.263970 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7e2a4-8fb7-4735-9e4d-7ce3905950de admin admin] PATH_INFO: `/v3/projects/ec8e48464f6a4d1886aaf89cc8b28e5f/users/d20025ab12f4450da20c8e41b931b5fc/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.276679 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 797/1599] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:17.283050 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02723d59-28e4-4807-beb6-9ebfd076939a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.283538 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02723d59-28e4-4807-beb6-9ebfd076939a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.283674 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb7e2a4-8fb7-4735-9e4d-7ce3905950de admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec8e48464f6a4d1886aaf89cc8b28e5f, user_id=d20025ab12f4450da20c8e41b931b5fc, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:17.283850 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02723d59-28e4-4807-beb6-9ebfd076939a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.287440 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb7e2a4-8fb7-4735-9e4d-7ce3905950de admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:17.305986 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 803/1600] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:17 2022] PUT /identity/v3/projects/ec8e48464f6a4d1886aaf89cc8b28e5f/users/d20025ab12f4450da20c8e41b931b5fc/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:17.309706 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7e2a4-8fb7-4735-9e4d-7ce3905950de admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:17.323154 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-02723d59-28e4-4807-beb6-9ebfd076939a None None] MFA Rules not processed for user `f6efd5bf855c47fe8ea69f4944bd2d0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:17.324567 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-02723d59-28e4-4807-beb6-9ebfd076939a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:17.339267 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:17.365766 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:17.379485 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7e2a4-8fb7-4735-9e4d-7ce3905950de admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:17.387252 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3a7e7-7dfc-401f-8f2d-ef704e8f8aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:17.388183 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3a7e7-7dfc-401f-8f2d-ef704e8f8aaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.388521 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3a7e7-7dfc-401f-8f2d-ef704e8f8aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.388864 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3a7e7-7dfc-401f-8f2d-ef704e8f8aaf admin admin] PATH_INFO: `/v3/projects/9e5535c0d612453ab394b0d90b0e6e02/users/14c980a621c4497a81b3cac3e811fb2e/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.404836 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3a7e7-7dfc-401f-8f2d-ef704e8f8aaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e5535c0d612453ab394b0d90b0e6e02, user_id=14c980a621c4497a81b3cac3e811fb2e, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:17.409402 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 798/1601] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:17.409402 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02723d59-28e4-4807-beb6-9ebfd076939a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:17.410007 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3a7e7-7dfc-401f-8f2d-ef704e8f8aaf admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:17.420457 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02723d59-28e4-4807-beb6-9ebfd076939a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:17.431618 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 804/1602] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:17 2022] PUT /identity/v3/projects/9e5535c0d612453ab394b0d90b0e6e02/users/14c980a621c4497a81b3cac3e811fb2e/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:17.431618 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3a7e7-7dfc-401f-8f2d-ef704e8f8aaf admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:17.496197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c716e8-36b8-44a6-a216-6fe1aeb0c975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:17.497200 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c716e8-36b8-44a6-a216-6fe1aeb0c975 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.497487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c716e8-36b8-44a6-a216-6fe1aeb0c975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.497762 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c716e8-36b8-44a6-a216-6fe1aeb0c975 admin admin] PATH_INFO: `/v3/projects/5da54606acd24962be630e28ff636d72/users/936db0436820496ebf8cc741852dda10/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.509545 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3a7e7-7dfc-401f-8f2d-ef704e8f8aaf admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:17.511909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49bc98-8d06-4304-9bca-902b0c546c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:17.512196 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49bc98-8d06-4304-9bca-902b0c546c18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.512479 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49bc98-8d06-4304-9bca-902b0c546c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.512748 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49bc98-8d06-4304-9bca-902b0c546c18 admin admin] PATH_INFO: `/v3/projects/798b460fbb2e4be69df7f5f419844832/users/6592aab6435242dca77b854a74ced4f2/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.533311 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49bc98-8d06-4304-9bca-902b0c546c18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=798b460fbb2e4be69df7f5f419844832, user_id=6592aab6435242dca77b854a74ced4f2, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:17.534494 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c716e8-36b8-44a6-a216-6fe1aeb0c975 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5da54606acd24962be630e28ff636d72, user_id=936db0436820496ebf8cc741852dda10, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:17.536174 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49bc98-8d06-4304-9bca-902b0c546c18 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:17.537740 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c716e8-36b8-44a6-a216-6fe1aeb0c975 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:17.555349 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 805/1603] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:17 2022] PUT /identity/v3/projects/798b460fbb2e4be69df7f5f419844832/users/6592aab6435242dca77b854a74ced4f2/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:17.556436 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49bc98-8d06-4304-9bca-902b0c546c18 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:17.570378 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 799/1604] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:17 2022] PUT /identity/v3/projects/5da54606acd24962be630e28ff636d72/users/936db0436820496ebf8cc741852dda10/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:17.571841 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c716e8-36b8-44a6-a216-6fe1aeb0c975 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:17.647353 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49bc98-8d06-4304-9bca-902b0c546c18 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:17.652925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816bc44-cd61-4187-977b-74d4757417bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:17.653955 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816bc44-cd61-4187-977b-74d4757417bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.654244 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816bc44-cd61-4187-977b-74d4757417bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.654521 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816bc44-cd61-4187-977b-74d4757417bb admin admin] PATH_INFO: `/v3/projects/59eff67373ac4a7091ee20065a20bea2/users/8e440699aceb4636bb68e796e811e758/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.661668 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c716e8-36b8-44a6-a216-6fe1aeb0c975 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:17.671818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac1947-d226-4bda-9a1a-53025730eebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:17.672928 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac1947-d226-4bda-9a1a-53025730eebd admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.673213 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac1947-d226-4bda-9a1a-53025730eebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.673490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac1947-d226-4bda-9a1a-53025730eebd admin admin] PATH_INFO: `/v3/projects/95161229e3f14e42857866e55b39ec09/users/7411441cb9894116b5ec11dd1881146f/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.674705 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d816bc44-cd61-4187-977b-74d4757417bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=59eff67373ac4a7091ee20065a20bea2, user_id=8e440699aceb4636bb68e796e811e758, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:17.677467 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d816bc44-cd61-4187-977b-74d4757417bb admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:17.692342 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac1947-d226-4bda-9a1a-53025730eebd admin admin] RBAC: Authorizing `identity:create_grant(project_id=95161229e3f14e42857866e55b39ec09, user_id=7411441cb9894116b5ec11dd1881146f, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:17.695001 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac1947-d226-4bda-9a1a-53025730eebd admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:17.699286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 806/1605] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:17 2022] PUT /identity/v3/projects/59eff67373ac4a7091ee20065a20bea2/users/8e440699aceb4636bb68e796e811e758/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:17.701111 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b7b592-a9d0-4e0f-a0e4-6d05f8e1eca7 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.701755 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b7b592-a9d0-4e0f-a0e4-6d05f8e1eca7 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.702268 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b7b592-a9d0-4e0f-a0e4-6d05f8e1eca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.713993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 800/1606] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:17 2022] PUT /identity/v3/projects/95161229e3f14e42857866e55b39ec09/users/7411441cb9894116b5ec11dd1881146f/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:17.714942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac1947-d226-4bda-9a1a-53025730eebd admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:17.723530 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-39b7b592-a9d0-4e0f-a0e4-6d05f8e1eca7 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:17.724864 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-39b7b592-a9d0-4e0f-a0e4-6d05f8e1eca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:17.741462 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:17.768485 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:17.791497 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac1947-d226-4bda-9a1a-53025730eebd admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:17.796983 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ffa9ee-51f9-4ab5-a740-14219eb99a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:17.797779 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffa9ee-51f9-4ab5-a740-14219eb99a84 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.798191 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffa9ee-51f9-4ab5-a740-14219eb99a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.798313 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffa9ee-51f9-4ab5-a740-14219eb99a84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.799024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ffa9ee-51f9-4ab5-a740-14219eb99a84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:17.800697 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 807/1607] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:17.801012 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ffa9ee-51f9-4ab5-a740-14219eb99a84 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:17.802159 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44552f-b865-4ae3-a9f5-f8f2d5647662 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.802463 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44552f-b865-4ae3-a9f5-f8f2d5647662 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.802797 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44552f-b865-4ae3-a9f5-f8f2d5647662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.828519 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 801/1608] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:17 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:17.831895 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8808f9e-a9a6-4976-bb54-0a026fbf04a3 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.831895 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8808f9e-a9a6-4976-bb54-0a026fbf04a3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.831895 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8808f9e-a9a6-4976-bb54-0a026fbf04a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.833987 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-cd44552f-b865-4ae3-a9f5-f8f2d5647662 None None] MFA Rules not processed for user `14c980a621c4497a81b3cac3e811fb2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:17.835597 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-cd44552f-b865-4ae3-a9f5-f8f2d5647662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:17.848039 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e8808f9e-a9a6-4976-bb54-0a026fbf04a3 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:17.849235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e8808f9e-a9a6-4976-bb54-0a026fbf04a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:17.901677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 808/1609] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:17 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:17.903182 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8155e0-8dcc-470e-b114-3ee3fdffb550 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.903502 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8155e0-8dcc-470e-b114-3ee3fdffb550 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.903786 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8155e0-8dcc-470e-b114-3ee3fdffb550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.917722 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 802/1610] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:17.917722 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13241276-8f82-472b-a8d3-f3aa3a88c4ad None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.917722 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13241276-8f82-472b-a8d3-f3aa3a88c4ad None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.917722 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13241276-8f82-472b-a8d3-f3aa3a88c4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.924610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-af8155e0-8dcc-470e-b114-3ee3fdffb550 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:17.925913 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-af8155e0-8dcc-470e-b114-3ee3fdffb550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:17.934043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 809/1611] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:17.935277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8933991-83f9-4e88-93bd-fa9ae8b37913 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.935662 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8933991-83f9-4e88-93bd-fa9ae8b37913 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.936017 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8933991-83f9-4e88-93bd-fa9ae8b37913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.948893 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-13241276-8f82-472b-a8d3-f3aa3a88c4ad None None] MFA Rules not processed for user `0cb00f50054442749dc681eb800e8117`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:17.950295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-13241276-8f82-472b-a8d3-f3aa3a88c4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:17.965230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-d8933991-83f9-4e88-93bd-fa9ae8b37913 None None] MFA Rules not processed for user `8e440699aceb4636bb68e796e811e758`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:17.966440 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d8933991-83f9-4e88-93bd-fa9ae8b37913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:18.025810 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 803/1612] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:18.026847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb68118-db44-40e2-857d-e7f8124c60ff None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.027139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb68118-db44-40e2-857d-e7f8124c60ff None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.027450 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb68118-db44-40e2-857d-e7f8124c60ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.039455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 810/1613] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:18.039455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8933991-83f9-4e88-93bd-fa9ae8b37913 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:18.054166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-fbb68118-db44-40e2-857d-e7f8124c60ff None None] MFA Rules not processed for user `7411441cb9894116b5ec11dd1881146f`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:18.055411 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fbb68118-db44-40e2-857d-e7f8124c60ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:18.083828 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:18.108322 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8933991-83f9-4e88-93bd-fa9ae8b37913 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:18.112493 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e335e35f-3437-49c8-bde6-9ba785231850 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:18.113258 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335e35f-3437-49c8-bde6-9ba785231850 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.113527 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335e35f-3437-49c8-bde6-9ba785231850 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.113799 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335e35f-3437-49c8-bde6-9ba785231850 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.120753 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:18.123470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e335e35f-3437-49c8-bde6-9ba785231850 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:18.126380 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e335e35f-3437-49c8-bde6-9ba785231850 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:18.128184 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 804/1614] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:18.129013 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb68118-db44-40e2-857d-e7f8124c60ff None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:18.137994 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 811/1615] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:18.139086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e335e35f-3437-49c8-bde6-9ba785231850 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:18.139515 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb68118-db44-40e2-857d-e7f8124c60ff None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:18.145523 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10689bd4-4116-4d00-bbc8-b0f1d22e9c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:18.147299 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10689bd4-4116-4d00-bbc8-b0f1d22e9c90 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.147548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10689bd4-4116-4d00-bbc8-b0f1d22e9c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.148215 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10689bd4-4116-4d00-bbc8-b0f1d22e9c90 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.148936 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10689bd4-4116-4d00-bbc8-b0f1d22e9c90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:18.150729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e335e35f-3437-49c8-bde6-9ba785231850 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:18.154750 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10689bd4-4116-4d00-bbc8-b0f1d22e9c90 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:18.161085 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3dcba-06be-41a1-aa48-61a4f3c582e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:18.165977 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3dcba-06be-41a1-aa48-61a4f3c582e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.165977 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3dcba-06be-41a1-aa48-61a4f3c582e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.165977 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3dcba-06be-41a1-aa48-61a4f3c582e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.165977 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de3dcba-06be-41a1-aa48-61a4f3c582e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:18.165977 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de3dcba-06be-41a1-aa48-61a4f3c582e7 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:18.183734 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 805/1616] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:18 2022] POST /identity/v3/users => generated 300 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:18.184857 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10689bd4-4116-4d00-bbc8-b0f1d22e9c90 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:18.195771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10689bd4-4116-4d00-bbc8-b0f1d22e9c90 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:18.199677 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbbd42b-ba4a-4d91-8d08-16ab943a208a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:18.200329 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbd42b-ba4a-4d91-8d08-16ab943a208a service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.200625 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbd42b-ba4a-4d91-8d08-16ab943a208a service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.200901 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbd42b-ba4a-4d91-8d08-16ab943a208a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.202919 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 812/1617] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:18 2022] POST /identity/v3/projects => generated 369 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:18.204093 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3dcba-06be-41a1-aa48-61a4f3c582e7 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:18.208462 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2de3dcba-06be-41a1-aa48-61a4f3c582e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:18.216069 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbbd42b-ba4a-4d91-8d08-16ab943a208a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:18.217519 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbbd42b-ba4a-4d91-8d08-16ab943a208a service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:18.229269 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 806/1618] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:18.229932 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbbd42b-ba4a-4d91-8d08-16ab943a208a service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:18.239974 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbbd42b-ba4a-4d91-8d08-16ab943a208a service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:18.282477 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3dcba-06be-41a1-aa48-61a4f3c582e7 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:18.287473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430d957-cd6e-4041-be39-1377583529cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:18.288189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430d957-cd6e-4041-be39-1377583529cd service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.288546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430d957-cd6e-4041-be39-1377583529cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.288826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430d957-cd6e-4041-be39-1377583529cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.313936 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6afe3f-d3b2-4ac0-96fe-943a8cc67ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:18.314632 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6afe3f-d3b2-4ac0-96fe-943a8cc67ea1 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.314913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6afe3f-d3b2-4ac0-96fe-943a8cc67ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.315189 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6afe3f-d3b2-4ac0-96fe-943a8cc67ea1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.315907 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6afe3f-d3b2-4ac0-96fe-943a8cc67ea1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:18.317809 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6afe3f-d3b2-4ac0-96fe-943a8cc67ea1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:18.346966 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 807/1619] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:18 2022] POST /identity/v3/projects => generated 403 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:18.348505 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6a5bf-271d-4776-93fd-916b7f383e82 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.348900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6a5bf-271d-4776-93fd-916b7f383e82 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.349243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6a5bf-271d-4776-93fd-916b7f383e82 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.350945 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 808/1620] 10.222.0.81 () {54 vars in 961 bytes} [Tue Jun 7 14:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:18.352564 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbc4ff-25fb-43fc-9dfe-283500e9ddd7 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.353135 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbc4ff-25fb-43fc-9dfe-283500e9ddd7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.353786 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbc4ff-25fb-43fc-9dfe-283500e9ddd7 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.355170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 809/1621] 10.222.0.81 () {54 vars in 961 bytes} [Tue Jun 7 14:21:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:18.356407 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbc4ff-25fb-43fc-9dfe-283500e9ddd7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:18.367877 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d430d957-cd6e-4041-be39-1377583529cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:18.369476 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d430d957-cd6e-4041-be39-1377583529cd service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:18.380494 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 813/1622] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:18.390691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430d957-cd6e-4041-be39-1377583529cd service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:18.408787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430d957-cd6e-4041-be39-1377583529cd service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:18.471134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbc4ff-25fb-43fc-9dfe-283500e9ddd7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:18.483555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab72e59-5d54-4774-9bab-2aa38f4a0fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:18.484948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab72e59-5d54-4774-9bab-2aa38f4a0fe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.485088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434810d-82bd-4492-98ea-64e76d527e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:18.485604 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab72e59-5d54-4774-9bab-2aa38f4a0fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.485990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab72e59-5d54-4774-9bab-2aa38f4a0fe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.486289 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434810d-82bd-4492-98ea-64e76d527e77 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.486674 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434810d-82bd-4492-98ea-64e76d527e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.486958 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434810d-82bd-4492-98ea-64e76d527e77 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.488023 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1434810d-82bd-4492-98ea-64e76d527e77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:18.488462 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab72e59-5d54-4774-9bab-2aa38f4a0fe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:18.490364 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1434810d-82bd-4492-98ea-64e76d527e77 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:18.490654 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab72e59-5d54-4774-9bab-2aa38f4a0fe3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:18.502681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 810/1623] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:18.506527 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab72e59-5d54-4774-9bab-2aa38f4a0fe3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:18.520926 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab72e59-5d54-4774-9bab-2aa38f4a0fe3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:18.522064 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 814/1624] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:18 2022] POST /identity/v3/users => generated 291 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:18.523753 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f989f27-46ac-4f50-aae4-b70f20d82b4d None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.524206 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f989f27-46ac-4f50-aae4-b70f20d82b4d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.524517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f989f27-46ac-4f50-aae4-b70f20d82b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.532250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20cbaad-0c3a-4dca-a110-a9a1c6cadb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:18.533525 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cbaad-0c3a-4dca-a110-a9a1c6cadb4e admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.534332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cbaad-0c3a-4dca-a110-a9a1c6cadb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.535012 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cbaad-0c3a-4dca-a110-a9a1c6cadb4e admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.536195 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20cbaad-0c3a-4dca-a110-a9a1c6cadb4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:18.539490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20cbaad-0c3a-4dca-a110-a9a1c6cadb4e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:18.544636 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-2f989f27-46ac-4f50-aae4-b70f20d82b4d None None] MFA Rules not processed for user `2ae726fbc9cf40e68c7587552a118a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:18.545843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2f989f27-46ac-4f50-aae4-b70f20d82b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:18.558482 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:18.564320 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:18.577009 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 811/1625] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:18 2022] POST /identity/v3/users => generated 308 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:18.578923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36179d53-77a6-43f7-b540-18960b0ce50c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.579230 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36179d53-77a6-43f7-b540-18960b0ce50c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.579523 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36179d53-77a6-43f7-b540-18960b0ce50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.604006 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-36179d53-77a6-43f7-b540-18960b0ce50c None None] MFA Rules not processed for user `2ae726fbc9cf40e68c7587552a118a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:18.607057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-36179d53-77a6-43f7-b540-18960b0ce50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:18.659414 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 815/1626] 10.222.0.81 () {58 vars in 1058 bytes} [Tue Jun 7 14:21:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:18.660555 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f989f27-46ac-4f50-aae4-b70f20d82b4d None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:18.674676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f989f27-46ac-4f50-aae4-b70f20d82b4d None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:18.683658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd001f7-a1fe-4bbe-9d9e-7e9488795dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:18.684786 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd001f7-a1fe-4bbe-9d9e-7e9488795dce admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.685142 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd001f7-a1fe-4bbe-9d9e-7e9488795dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.685748 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd001f7-a1fe-4bbe-9d9e-7e9488795dce admin admin] PATH_INFO: `/v3/projects/b284a7c77bd04bb38bc9dd74c3c6a22b/users/9b733cc0952b4cc59b55f4c587040d88/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.714640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 812/1627] 10.222.0.81 () {58 vars in 1058 bytes} [Tue Jun 7 14:21:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:18.715525 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36179d53-77a6-43f7-b540-18960b0ce50c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:18.723469 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd001f7-a1fe-4bbe-9d9e-7e9488795dce admin admin] RBAC: Authorizing `identity:create_grant(project_id=b284a7c77bd04bb38bc9dd74c3c6a22b, user_id=9b733cc0952b4cc59b55f4c587040d88, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:18.726254 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd001f7-a1fe-4bbe-9d9e-7e9488795dce admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:18.735368 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36179d53-77a6-43f7-b540-18960b0ce50c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:18.743413 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 816/1628] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:18 2022] PUT /identity/v3/projects/b284a7c77bd04bb38bc9dd74c3c6a22b/users/9b733cc0952b4cc59b55f4c587040d88/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:18.744321 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd001f7-a1fe-4bbe-9d9e-7e9488795dce admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:18.822713 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd001f7-a1fe-4bbe-9d9e-7e9488795dce admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:18.829323 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f622eb-d489-49b6-8968-347d1dd33fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:18.830006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f622eb-d489-49b6-8968-347d1dd33fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.830287 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f622eb-d489-49b6-8968-347d1dd33fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.830561 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f622eb-d489-49b6-8968-347d1dd33fe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.831384 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f622eb-d489-49b6-8968-347d1dd33fe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:18.833517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f622eb-d489-49b6-8968-347d1dd33fe2 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:18.850063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 817/1629] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:18.850063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f622eb-d489-49b6-8968-347d1dd33fe2 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:18.876509 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b149f6-7493-4e78-bed0-1c351d828b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:18.877612 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b149f6-7493-4e78-bed0-1c351d828b7e admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.879417 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b149f6-7493-4e78-bed0-1c351d828b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.880467 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b149f6-7493-4e78-bed0-1c351d828b7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.882891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b149f6-7493-4e78-bed0-1c351d828b7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:18.891110 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b149f6-7493-4e78-bed0-1c351d828b7e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:18.918812 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 813/1630] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:18.919911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b149f6-7493-4e78-bed0-1c351d828b7e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:18.944355 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f622eb-d489-49b6-8968-347d1dd33fe2 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:18.951576 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c21a02-2f1e-465d-a753-e8c27816b96f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:18.952238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c21a02-2f1e-465d-a753-e8c27816b96f service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.953226 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c21a02-2f1e-465d-a753-e8c27816b96f service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.953700 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c21a02-2f1e-465d-a753-e8c27816b96f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.972705 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:19.041711 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c21a02-2f1e-465d-a753-e8c27816b96f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:19.043521 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c21a02-2f1e-465d-a753-e8c27816b96f service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:19.056944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 818/1631] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:19.058162 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c21a02-2f1e-465d-a753-e8c27816b96f service nova] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:19.072050 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c21a02-2f1e-465d-a753-e8c27816b96f service nova] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:19.078288 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59def075-019b-43c0-8842-01b6ea443ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:19.078945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59def075-019b-43c0-8842-01b6ea443ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:19.079375 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59def075-019b-43c0-8842-01b6ea443ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:19.079665 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59def075-019b-43c0-8842-01b6ea443ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:19.080405 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59def075-019b-43c0-8842-01b6ea443ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:19.082171 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59def075-019b-43c0-8842-01b6ea443ff8 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:19.087200 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b149f6-7493-4e78-bed0-1c351d828b7e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:19.099447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 819/1632] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:19.099447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59def075-019b-43c0-8842-01b6ea443ff8 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:19.125073 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c60ceb-a163-48ae-9217-5997b11ec551 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:19.125843 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c60ceb-a163-48ae-9217-5997b11ec551 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:19.126108 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c60ceb-a163-48ae-9217-5997b11ec551 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:19.126218 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59def075-019b-43c0-8842-01b6ea443ff8 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:19.126543 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c60ceb-a163-48ae-9217-5997b11ec551 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:19.132862 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95760a68-7c92-4996-92d4-1f332c166be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:19.134434 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95760a68-7c92-4996-92d4-1f332c166be6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:19.134767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95760a68-7c92-4996-92d4-1f332c166be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:19.135024 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95760a68-7c92-4996-92d4-1f332c166be6 admin admin] PATH_INFO: `/v3/projects/99ac6fdcef584017ad990af7f33c7033/users/d7192f815ea44e7f806c1b8d487711d9/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:19.193921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:19.195838 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95760a68-7c92-4996-92d4-1f332c166be6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ac6fdcef584017ad990af7f33c7033, user_id=d7192f815ea44e7f806c1b8d487711d9, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:19.198741 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95760a68-7c92-4996-92d4-1f332c166be6 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:19.242282 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 820/1633] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:19 2022] PUT /identity/v3/projects/99ac6fdcef584017ad990af7f33c7033/users/d7192f815ea44e7f806c1b8d487711d9/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:19.246220 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95760a68-7c92-4996-92d4-1f332c166be6 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:19.251801 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c60ceb-a163-48ae-9217-5997b11ec551 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:19.253761 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c60ceb-a163-48ae-9217-5997b11ec551 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:19.267858 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 814/1634] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:19.269138 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c60ceb-a163-48ae-9217-5997b11ec551 service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:19.285316 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c60ceb-a163-48ae-9217-5997b11ec551 service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:19.365582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3eec28-05f2-4742-8276-4d8495342e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:19.367041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3eec28-05f2-4742-8276-4d8495342e8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:19.367358 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3eec28-05f2-4742-8276-4d8495342e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:19.367926 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3eec28-05f2-4742-8276-4d8495342e8b admin admin] PATH_INFO: `/v3/projects/b284a7c77bd04bb38bc9dd74c3c6a22b/users/9b733cc0952b4cc59b55f4c587040d88/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:19.370829 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3eec28-05f2-4742-8276-4d8495342e8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b284a7c77bd04bb38bc9dd74c3c6a22b, user_id=9b733cc0952b4cc59b55f4c587040d88, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:19.374177 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3eec28-05f2-4742-8276-4d8495342e8b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:19.398274 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 815/1635] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:19 2022] PUT /identity/v3/projects/b284a7c77bd04bb38bc9dd74c3c6a22b/users/9b733cc0952b4cc59b55f4c587040d88/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:19.398274 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef753362-e764-4e55-98c4-3072c773afbf None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:19.398274 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef753362-e764-4e55-98c4-3072c773afbf None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:19.398274 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef753362-e764-4e55-98c4-3072c773afbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:19.398872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95760a68-7c92-4996-92d4-1f332c166be6 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:19.429395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ef753362-e764-4e55-98c4-3072c773afbf None None] MFA Rules not processed for user `936db0436820496ebf8cc741852dda10`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:19.431146 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ef753362-e764-4e55-98c4-3072c773afbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:19.542881 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 816/1636] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:19.553450 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef753362-e764-4e55-98c4-3072c773afbf None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:19.558420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c9151-48f1-4c18-9ecc-2b982f531eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:19.559372 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c9151-48f1-4c18-9ecc-2b982f531eaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:19.559662 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c9151-48f1-4c18-9ecc-2b982f531eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:19.559991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c9151-48f1-4c18-9ecc-2b982f531eaa admin admin] PATH_INFO: `/v3/projects/19af625105d842adbe8b07a4a0f44118/users/f04282fce71d49e29fdff413ba73f7a2/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:19.595713 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c9151-48f1-4c18-9ecc-2b982f531eaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=19af625105d842adbe8b07a4a0f44118, user_id=f04282fce71d49e29fdff413ba73f7a2, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:19.598493 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c9151-48f1-4c18-9ecc-2b982f531eaa admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:19.646060 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 821/1637] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:19 2022] PUT /identity/v3/projects/19af625105d842adbe8b07a4a0f44118/users/f04282fce71d49e29fdff413ba73f7a2/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:19.648738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac2a93-6261-4de5-b14b-9961afed0e67 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:19.649093 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac2a93-6261-4de5-b14b-9961afed0e67 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:19.649377 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac2a93-6261-4de5-b14b-9961afed0e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:19.710879 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:19.712375 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:19.739124 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-f7ac2a93-6261-4de5-b14b-9961afed0e67 None None] MFA Rules not processed for user `9b733cc0952b4cc59b55f4c587040d88`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:19.741562 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f7ac2a93-6261-4de5-b14b-9961afed0e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:19.746025 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef753362-e764-4e55-98c4-3072c773afbf None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:19.875213 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f96c22-d750-40d6-bbd0-5b75a4a9450d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:19.876344 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f96c22-d750-40d6-bbd0-5b75a4a9450d admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:19.876857 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f96c22-d750-40d6-bbd0-5b75a4a9450d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:19.877352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f96c22-d750-40d6-bbd0-5b75a4a9450d admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:19.878311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f96c22-d750-40d6-bbd0-5b75a4a9450d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:19.880251 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f96c22-d750-40d6-bbd0-5b75a4a9450d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:19.899548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 817/1638] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:19.901437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626ec44-3bd2-4e34-b074-b77306822ace None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:19.901955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626ec44-3bd2-4e34-b074-b77306822ace None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:19.902473 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626ec44-3bd2-4e34-b074-b77306822ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:19.916581 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 822/1639] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:19.917581 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac2a93-6261-4de5-b14b-9961afed0e67 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:19.929696 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4626ec44-3bd2-4e34-b074-b77306822ace None None] MFA Rules not processed for user `936db0436820496ebf8cc741852dda10`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:19.931190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4626ec44-3bd2-4e34-b074-b77306822ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:19.939969 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac2a93-6261-4de5-b14b-9961afed0e67 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:19.953769 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ea868-785c-42ad-878a-bc9f1d32de5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:19.954630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ea868-785c-42ad-878a-bc9f1d32de5a admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:19.955016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ea868-785c-42ad-878a-bc9f1d32de5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:19.955665 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ea868-785c-42ad-878a-bc9f1d32de5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:19.956738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ea868-785c-42ad-878a-bc9f1d32de5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:19.959147 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ea868-785c-42ad-878a-bc9f1d32de5a admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:19.982423 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 823/1640] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:19.986137 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cede123-cd6e-41c1-b065-ea534e4ba95d None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:19.986608 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cede123-cd6e-41c1-b065-ea534e4ba95d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:19.988966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cede123-cd6e-41c1-b065-ea534e4ba95d None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:19.993362 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 824/1641] 10.222.0.81 () {54 vars in 912 bytes} [Tue Jun 7 14:21:19 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:19.994831 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e23d2-2869-4a6d-8b4e-83c69bdc5c8c None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:19.996463 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e23d2-2869-4a6d-8b4e-83c69bdc5c8c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:19.996855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e23d2-2869-4a6d-8b4e-83c69bdc5c8c None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:19.998510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 825/1642] 10.222.0.81 () {54 vars in 912 bytes} [Tue Jun 7 14:21:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:20.005100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e23d2-2869-4a6d-8b4e-83c69bdc5c8c None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:20.035555 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e23d2-2869-4a6d-8b4e-83c69bdc5c8c None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:20.045486 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a034b1-08d4-4334-8d26-77b3d87da26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:20.047492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a034b1-08d4-4334-8d26-77b3d87da26e admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:20.048468 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a034b1-08d4-4334-8d26-77b3d87da26e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:20.048855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a034b1-08d4-4334-8d26-77b3d87da26e admin admin] PATH_INFO: `/v3/projects/99ac6fdcef584017ad990af7f33c7033/users/d7192f815ea44e7f806c1b8d487711d9/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:20.053775 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a034b1-08d4-4334-8d26-77b3d87da26e admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ac6fdcef584017ad990af7f33c7033, user_id=d7192f815ea44e7f806c1b8d487711d9, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:20.058833 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a034b1-08d4-4334-8d26-77b3d87da26e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:20.067070 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 818/1643] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:20.068595 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626ec44-3bd2-4e34-b074-b77306822ace None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:20.087432 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 826/1644] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:19 2022] PUT /identity/v3/projects/99ac6fdcef584017ad990af7f33c7033/users/d7192f815ea44e7f806c1b8d487711d9/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:20.088681 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a034b1-08d4-4334-8d26-77b3d87da26e admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:20.188275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626ec44-3bd2-4e34-b074-b77306822ace None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:20.232287 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a034b1-08d4-4334-8d26-77b3d87da26e admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:20.247528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e5292-6a06-48a8-a43d-0427f2cf252b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:20.248332 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e5292-6a06-48a8-a43d-0427f2cf252b admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:20.248618 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e5292-6a06-48a8-a43d-0427f2cf252b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:20.248888 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e5292-6a06-48a8-a43d-0427f2cf252b admin admin] PATH_INFO: `/v3/projects/19af625105d842adbe8b07a4a0f44118/users/f04282fce71d49e29fdff413ba73f7a2/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:20.251926 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e5292-6a06-48a8-a43d-0427f2cf252b admin admin] RBAC: Authorizing `identity:create_grant(project_id=19af625105d842adbe8b07a4a0f44118, user_id=f04282fce71d49e29fdff413ba73f7a2, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:20.255028 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e5292-6a06-48a8-a43d-0427f2cf252b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:20.288824 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 827/1645] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:20 2022] PUT /identity/v3/projects/19af625105d842adbe8b07a4a0f44118/users/f04282fce71d49e29fdff413ba73f7a2/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:20.288824 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63642d38-8348-446e-92ec-d502aa5fa85c None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:20.288824 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63642d38-8348-446e-92ec-d502aa5fa85c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:20.288824 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63642d38-8348-446e-92ec-d502aa5fa85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:20.297435 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0a287-a2c8-484f-bbb0-2c96b1155dde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:20.298382 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0a287-a2c8-484f-bbb0-2c96b1155dde service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:20.299199 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0a287-a2c8-484f-bbb0-2c96b1155dde service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:20.299618 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0a287-a2c8-484f-bbb0-2c96b1155dde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:20.307707 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:20.318710 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-63642d38-8348-446e-92ec-d502aa5fa85c None None] MFA Rules not processed for user `2ae726fbc9cf40e68c7587552a118a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:20.320397 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-63642d38-8348-446e-92ec-d502aa5fa85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:20.400859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:20.419722 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 828/1646] 10.222.0.81 () {58 vars in 1009 bytes} [Tue Jun 7 14:21:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:20.422654 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b342d-4605-4040-8938-5e92d69bac2e None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:20.423064 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b342d-4605-4040-8938-5e92d69bac2e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:20.423861 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b342d-4605-4040-8938-5e92d69bac2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:20.450960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b0a287-a2c8-484f-bbb0-2c96b1155dde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:20.451281 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-0b0b342d-4605-4040-8938-5e92d69bac2e None None] MFA Rules not processed for user `2ae726fbc9cf40e68c7587552a118a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:20.452977 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0b0b342d-4605-4040-8938-5e92d69bac2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:20.454764 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b0a287-a2c8-484f-bbb0-2c96b1155dde service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:20.463891 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 829/1647] 10.222.0.81 () {58 vars in 1009 bytes} [Tue Jun 7 14:21:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:20.465390 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611659c3-9964-4582-92a0-55759e843d77 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:20.466122 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611659c3-9964-4582-92a0-55759e843d77 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:20.466501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611659c3-9964-4582-92a0-55759e843d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:20.478892 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 819/1648] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:20.480733 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0a287-a2c8-484f-bbb0-2c96b1155dde service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:20.491463 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0a287-a2c8-484f-bbb0-2c96b1155dde service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:20.519925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-611659c3-9964-4582-92a0-55759e843d77 None None] MFA Rules not processed for user `f04282fce71d49e29fdff413ba73f7a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:20.521449 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-611659c3-9964-4582-92a0-55759e843d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:20.581674 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07153094-db31-4445-97e9-62821179e678 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:20.582633 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07153094-db31-4445-97e9-62821179e678 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:20.583021 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07153094-db31-4445-97e9-62821179e678 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:20.583535 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07153094-db31-4445-97e9-62821179e678 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:20.603011 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07153094-db31-4445-97e9-62821179e678 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:20.604870 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07153094-db31-4445-97e9-62821179e678 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:20.617560 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 830/1649] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:20 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:20.618500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611659c3-9964-4582-92a0-55759e843d77 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:20.628799 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 820/1650] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:20.630115 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07153094-db31-4445-97e9-62821179e678 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:20.641899 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07153094-db31-4445-97e9-62821179e678 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:20.651585 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd62116-df9d-497e-a22b-b897ce4d8b8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:20.655495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd62116-df9d-497e-a22b-b897ce4d8b8a service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:20.657976 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd62116-df9d-497e-a22b-b897ce4d8b8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:20.658925 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd62116-df9d-497e-a22b-b897ce4d8b8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:20.673032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd62116-df9d-497e-a22b-b897ce4d8b8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:20.674637 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd62116-df9d-497e-a22b-b897ce4d8b8a service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:20.687938 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 821/1651] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:20.712142 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611659c3-9964-4582-92a0-55759e843d77 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:20.723426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb3419-9296-42a6-bad4-0d86ccd9d63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:20.725671 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb3419-9296-42a6-bad4-0d86ccd9d63a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:20.726258 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb3419-9296-42a6-bad4-0d86ccd9d63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:20.726974 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb3419-9296-42a6-bad4-0d86ccd9d63a admin admin] PATH_INFO: `/v3/users/f6efd5bf855c47fe8ea69f4944bd2d0f` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:20.729447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb3419-9296-42a6-bad4-0d86ccd9d63a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6efd5bf855c47fe8ea69f4944bd2d0f)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:20.730821 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: WARNING py.warnings [None req-2bfb3419-9296-42a6-bad4-0d86ccd9d63a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:21:20.730821 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: warnings.warn(msg) Jun 07 14:21:20.730821 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: Jun 07 14:21:20.732488 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb3419-9296-42a6-bad4-0d86ccd9d63a admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:20.849650 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-2bfb3419-9296-42a6-bad4-0d86ccd9d63a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6efd5bf855c47fe8ea69f4944bd2d0f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:20.917844 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-2bfb3419-9296-42a6-bad4-0d86ccd9d63a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6efd5bf855c47fe8ea69f4944bd2d0f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:20.920020 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-2bfb3419-9296-42a6-bad4-0d86ccd9d63a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6efd5bf855c47fe8ea69f4944bd2d0f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:20.922899 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-2bfb3419-9296-42a6-bad4-0d86ccd9d63a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6efd5bf855c47fe8ea69f4944bd2d0f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:20.938551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd62116-df9d-497e-a22b-b897ce4d8b8a service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:20.947320 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd62116-df9d-497e-a22b-b897ce4d8b8a service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:20.958431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db58bdf-01be-4cf0-88ab-c39032bb4639 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:20.960175 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db58bdf-01be-4cf0-88ab-c39032bb4639 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:20.961013 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db58bdf-01be-4cf0-88ab-c39032bb4639 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:20.962059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db58bdf-01be-4cf0-88ab-c39032bb4639 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:20.966902 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2db58bdf-01be-4cf0-88ab-c39032bb4639 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:20.967467 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-2bfb3419-9296-42a6-bad4-0d86ccd9d63a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6efd5bf855c47fe8ea69f4944bd2d0f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:20.975576 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:21.000999 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 831/1652] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:21:20 2022] DELETE /identity/v3/users/f6efd5bf855c47fe8ea69f4944bd2d0f => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:21.000999 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb3419-9296-42a6-bad4-0d86ccd9d63a admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:21.001440 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2bfb3419-9296-42a6-bad4-0d86ccd9d63a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:21.009498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db58bdf-01be-4cf0-88ab-c39032bb4639 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:21.011356 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db58bdf-01be-4cf0-88ab-c39032bb4639 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:21.033630 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 822/1653] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:20 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:21:21.036146 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e15a5-9653-45b1-b7ce-30cb3c83bacb None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:21.043732 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e15a5-9653-45b1-b7ce-30cb3c83bacb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:21.044788 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e15a5-9653-45b1-b7ce-30cb3c83bacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:21.077131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-553e15a5-9653-45b1-b7ce-30cb3c83bacb None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:21.078546 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-553e15a5-9653-45b1-b7ce-30cb3c83bacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:21.222032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 823/1654] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:21.237961 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb3419-9296-42a6-bad4-0d86ccd9d63a admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:21.252231 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b5902-efef-4db5-883a-a745d4b75097 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:21.256785 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b5902-efef-4db5-883a-a745d4b75097 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:21.257226 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b5902-efef-4db5-883a-a745d4b75097 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:21.257508 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b5902-efef-4db5-883a-a745d4b75097 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:21.261952 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-349b5902-efef-4db5-883a-a745d4b75097 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:21.342042 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e15a5-9653-45b1-b7ce-30cb3c83bacb None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:21.344388 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-553e15a5-9653-45b1-b7ce-30cb3c83bacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:21.389337 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:21.463667 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b5902-efef-4db5-883a-a745d4b75097 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:21.466684 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b5902-efef-4db5-883a-a745d4b75097 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:21.496219 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e15a5-9653-45b1-b7ce-30cb3c83bacb None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:21.497383 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 832/1655] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:21.503291 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf571e2-4469-4e7f-b649-fafac59ad455 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:21.504036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf571e2-4469-4e7f-b649-fafac59ad455 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:21.504335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf571e2-4469-4e7f-b649-fafac59ad455 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:21.504664 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf571e2-4469-4e7f-b649-fafac59ad455 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:21.516320 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf571e2-4469-4e7f-b649-fafac59ad455 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:21.521006 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf571e2-4469-4e7f-b649-fafac59ad455 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:21.567574 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 824/1656] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:21:21.975355 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b5902-efef-4db5-883a-a745d4b75097 service nova] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:21.980199 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-349b5902-efef-4db5-883a-a745d4b75097 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:22.003190 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b5902-efef-4db5-883a-a745d4b75097 service nova] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:22.009229 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97fcc37-db21-4ffc-a615-da0fb6d74c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:22.010481 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fcc37-db21-4ffc-a615-da0fb6d74c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:22.010899 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fcc37-db21-4ffc-a615-da0fb6d74c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:22.011510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fcc37-db21-4ffc-a615-da0fb6d74c66 admin admin] PATH_INFO: `/v3/users/7411441cb9894116b5ec11dd1881146f` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:22.013271 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97fcc37-db21-4ffc-a615-da0fb6d74c66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7411441cb9894116b5ec11dd1881146f)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:22.016285 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97fcc37-db21-4ffc-a615-da0fb6d74c66 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:22.085177 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d97fcc37-db21-4ffc-a615-da0fb6d74c66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7411441cb9894116b5ec11dd1881146f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:22.105185 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d97fcc37-db21-4ffc-a615-da0fb6d74c66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7411441cb9894116b5ec11dd1881146f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:22.106092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d97fcc37-db21-4ffc-a615-da0fb6d74c66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7411441cb9894116b5ec11dd1881146f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:22.106852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d97fcc37-db21-4ffc-a615-da0fb6d74c66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7411441cb9894116b5ec11dd1881146f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:22.118887 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d97fcc37-db21-4ffc-a615-da0fb6d74c66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7411441cb9894116b5ec11dd1881146f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:22.141474 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 833/1657] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:21:21 2022] DELETE /identity/v3/users/7411441cb9894116b5ec11dd1881146f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:22.146150 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638cb29b-f64d-44da-b45d-5bbca0b0cf4d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:22.146719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638cb29b-f64d-44da-b45d-5bbca0b0cf4d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:22.146719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638cb29b-f64d-44da-b45d-5bbca0b0cf4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:22.215117 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-638cb29b-f64d-44da-b45d-5bbca0b0cf4d None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:22.215117 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-638cb29b-f64d-44da-b45d-5bbca0b0cf4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:22.383637 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 825/1658] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:22.430775 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97fcc37-db21-4ffc-a615-da0fb6d74c66 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:22.434796 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d97fcc37-db21-4ffc-a615-da0fb6d74c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:22.445566 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:22.560154 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a55138-81d9-42c7-88b3-03c6648135bf None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:22.560154 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a55138-81d9-42c7-88b3-03c6648135bf None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:22.560425 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a55138-81d9-42c7-88b3-03c6648135bf None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:22.564338 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 826/1659] 10.222.0.81 () {54 vars in 912 bytes} [Tue Jun 7 14:21:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:22.572902 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c2ed5-ff5f-416a-a2ba-dc44c3fbd450 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:22.573188 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c2ed5-ff5f-416a-a2ba-dc44c3fbd450 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:22.573468 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c2ed5-ff5f-416a-a2ba-dc44c3fbd450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:22.600432 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97fcc37-db21-4ffc-a615-da0fb6d74c66 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:22.615819 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6e027-9ca3-4d8d-bc5c-9cc53e932cc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:22.617975 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6e027-9ca3-4d8d-bc5c-9cc53e932cc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:22.619767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6e027-9ca3-4d8d-bc5c-9cc53e932cc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:22.620112 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6e027-9ca3-4d8d-bc5c-9cc53e932cc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:22.682114 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6e027-9ca3-4d8d-bc5c-9cc53e932cc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:22.688262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6e027-9ca3-4d8d-bc5c-9cc53e932cc9 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:22.742243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-606c2ed5-ff5f-416a-a2ba-dc44c3fbd450 None None] MFA Rules not processed for user `a6a08eaecbcd41228571739d8387882a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:22.761661 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 834/1660] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:22.770302 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-606c2ed5-ff5f-416a-a2ba-dc44c3fbd450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:22.791323 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 827/1661] 10.222.0.81 () {58 vars in 1009 bytes} [Tue Jun 7 14:21:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:21:22.808626 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6e027-9ca3-4d8d-bc5c-9cc53e932cc9 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:22.810701 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f6b6e027-9ca3-4d8d-bc5c-9cc53e932cc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:22.842247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6e027-9ca3-4d8d-bc5c-9cc53e932cc9 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:22.861530 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eff3fd-76f4-4ce7-bac2-dfde14803a9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:22.862208 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eff3fd-76f4-4ce7-bac2-dfde14803a9a service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:22.862492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eff3fd-76f4-4ce7-bac2-dfde14803a9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:22.862767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eff3fd-76f4-4ce7-bac2-dfde14803a9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:22.881417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eff3fd-76f4-4ce7-bac2-dfde14803a9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:22.882877 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eff3fd-76f4-4ce7-bac2-dfde14803a9a service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:22.903286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 835/1662] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:23.375824 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c2ed5-ff5f-416a-a2ba-dc44c3fbd450 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:23.379464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-606c2ed5-ff5f-416a-a2ba-dc44c3fbd450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:23.413010 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c2ed5-ff5f-416a-a2ba-dc44c3fbd450 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:23.441929 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a19b9c-b460-4290-b19a-1cd4353e8a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:23.445609 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a19b9c-b460-4290-b19a-1cd4353e8a7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:23.446250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a19b9c-b460-4290-b19a-1cd4353e8a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:23.446701 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a19b9c-b460-4290-b19a-1cd4353e8a7a admin admin] PATH_INFO: `/v3/projects/a7844e15ebe643e1b176d40b6cbf3ace` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:23.449841 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a19b9c-b460-4290-b19a-1cd4353e8a7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7844e15ebe643e1b176d40b6cbf3ace)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:23.453106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-22a19b9c-b460-4290-b19a-1cd4353e8a7a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:21:23.453106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: warnings.warn(msg) Jun 07 14:21:23.453106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: Jun 07 14:21:23.455401 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a19b9c-b460-4290-b19a-1cd4353e8a7a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:23.669902 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-22a19b9c-b460-4290-b19a-1cd4353e8a7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7844e15ebe643e1b176d40b6cbf3ace', 'request_id': 'req-22a19b9c-b460-4290-b19a-1cd4353e8a7a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:23.697726 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-22a19b9c-b460-4290-b19a-1cd4353e8a7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7844e15ebe643e1b176d40b6cbf3ace', 'request_id': 'req-22a19b9c-b460-4290-b19a-1cd4353e8a7a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:23.722838 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-22a19b9c-b460-4290-b19a-1cd4353e8a7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7844e15ebe643e1b176d40b6cbf3ace', 'request_id': 'req-22a19b9c-b460-4290-b19a-1cd4353e8a7a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:23.784338 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-22a19b9c-b460-4290-b19a-1cd4353e8a7a admin admin] The token cache is being invalidate because project a7844e15ebe643e1b176d40b6cbf3ace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:21:23.784649 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-22a19b9c-b460-4290-b19a-1cd4353e8a7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:23.787624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-22a19b9c-b460-4290-b19a-1cd4353e8a7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:23.789263 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 828/1663] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:21:23 2022] DELETE /identity/v3/projects/a7844e15ebe643e1b176d40b6cbf3ace => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:23.793618 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eff3fd-76f4-4ce7-bac2-dfde14803a9a service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:23.817373 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eff3fd-76f4-4ce7-bac2-dfde14803a9a service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:23.837731 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d25efa-b249-457f-b92d-b613043ce59f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:23.838027 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d25efa-b249-457f-b92d-b613043ce59f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:23.838309 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d25efa-b249-457f-b92d-b613043ce59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:23.908639 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-66d25efa-b249-457f-b92d-b613043ce59f None None] MFA Rules not processed for user `d7192f815ea44e7f806c1b8d487711d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:23.909890 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-66d25efa-b249-457f-b92d-b613043ce59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:23.942528 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:24.053596 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a808a-0a8b-4654-bc88-6a515873b12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:24.055676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a808a-0a8b-4654-bc88-6a515873b12c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:24.056494 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a808a-0a8b-4654-bc88-6a515873b12c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:24.057557 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a808a-0a8b-4654-bc88-6a515873b12c admin admin] PATH_INFO: `/v3/users/9b733cc0952b4cc59b55f4c587040d88` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:24.059339 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a808a-0a8b-4654-bc88-6a515873b12c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b733cc0952b4cc59b55f4c587040d88)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:24.064211 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a808a-0a8b-4654-bc88-6a515873b12c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:24.112866 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 829/1664] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:23 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:21:24.232012 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1b4a808a-0a8b-4654-bc88-6a515873b12c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b733cc0952b4cc59b55f4c587040d88'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:24.273015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1b4a808a-0a8b-4654-bc88-6a515873b12c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b733cc0952b4cc59b55f4c587040d88'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:24.273918 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1b4a808a-0a8b-4654-bc88-6a515873b12c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b733cc0952b4cc59b55f4c587040d88'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:24.275991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1b4a808a-0a8b-4654-bc88-6a515873b12c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b733cc0952b4cc59b55f4c587040d88'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:24.304558 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1b4a808a-0a8b-4654-bc88-6a515873b12c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b733cc0952b4cc59b55f4c587040d88'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:24.330786 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 836/1665] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:21:23 2022] DELETE /identity/v3/users/9b733cc0952b4cc59b55f4c587040d88 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:24.504940 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9c937-21d4-42a3-b301-25f38b474012 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:24.505468 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9c937-21d4-42a3-b301-25f38b474012 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:24.505736 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9c937-21d4-42a3-b301-25f38b474012 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:24.507531 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 830/1666] 10.222.0.81 () {54 vars in 960 bytes} [Tue Jun 7 14:21:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:24.513915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1020ba9-f3ac-4fae-8baa-12da58eea5b5 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:24.514348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1020ba9-f3ac-4fae-8baa-12da58eea5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:24.514785 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1020ba9-f3ac-4fae-8baa-12da58eea5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:24.562223 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-d1020ba9-f3ac-4fae-8baa-12da58eea5b5 None None] MFA Rules not processed for user `ab73cf3febbf4c768cd0116306f16c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:24.564931 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d1020ba9-f3ac-4fae-8baa-12da58eea5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:24.574840 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9c937-21d4-42a3-b301-25f38b474012 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:24.576852 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cbc9c937-21d4-42a3-b301-25f38b474012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:24.592062 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:24.732291 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9c937-21d4-42a3-b301-25f38b474012 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:24.743240 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02852a0c-62ef-4428-9998-cd2373bc4731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:24.744345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02852a0c-62ef-4428-9998-cd2373bc4731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:24.744947 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02852a0c-62ef-4428-9998-cd2373bc4731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:24.745053 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02852a0c-62ef-4428-9998-cd2373bc4731 admin admin] PATH_INFO: `/v3/users/14c980a621c4497a81b3cac3e811fb2e` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:24.749088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02852a0c-62ef-4428-9998-cd2373bc4731 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14c980a621c4497a81b3cac3e811fb2e)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:24.756317 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02852a0c-62ef-4428-9998-cd2373bc4731 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:24.794462 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 837/1667] 10.222.0.81 () {58 vars in 1057 bytes} [Tue Jun 7 14:21:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:24.806852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1020ba9-f3ac-4fae-8baa-12da58eea5b5 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:24.820911 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1020ba9-f3ac-4fae-8baa-12da58eea5b5 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:24.843762 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa207f-65fb-454f-b249-dbd1645bd5ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab73cf3febbf4c768cd0116306f16c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:24.844928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa207f-65fb-454f-b249-dbd1645bd5ab service placement] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:24.845244 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa207f-65fb-454f-b249-dbd1645bd5ab service placement] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:24.845530 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa207f-65fb-454f-b249-dbd1645bd5ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:24.856663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b2aa207f-65fb-454f-b249-dbd1645bd5ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:24.879748 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa207f-65fb-454f-b249-dbd1645bd5ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:24.881364 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa207f-65fb-454f-b249-dbd1645bd5ab service placement] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:24.889772 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-02852a0c-62ef-4428-9998-cd2373bc4731 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14c980a621c4497a81b3cac3e811fb2e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:24.892360 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-02852a0c-62ef-4428-9998-cd2373bc4731 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14c980a621c4497a81b3cac3e811fb2e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:24.897575 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-02852a0c-62ef-4428-9998-cd2373bc4731 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14c980a621c4497a81b3cac3e811fb2e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:24.903784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 838/1668] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:21:24.927794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-02852a0c-62ef-4428-9998-cd2373bc4731 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14c980a621c4497a81b3cac3e811fb2e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:24.965080 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-02852a0c-62ef-4428-9998-cd2373bc4731 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14c980a621c4497a81b3cac3e811fb2e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:24.991373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 831/1669] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:21:24 2022] DELETE /identity/v3/users/14c980a621c4497a81b3cac3e811fb2e => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:21:24.996051 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239950b2-3631-47d0-858b-ff76f56205ed None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:24.996329 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239950b2-3631-47d0-858b-ff76f56205ed None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:24.996615 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239950b2-3631-47d0-858b-ff76f56205ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:25.033804 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-239950b2-3631-47d0-858b-ff76f56205ed None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:25.039687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-239950b2-3631-47d0-858b-ff76f56205ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:25.188831 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 839/1670] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:25.224225 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02852a0c-62ef-4428-9998-cd2373bc4731 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:25.227907 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-02852a0c-62ef-4428-9998-cd2373bc4731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:25.245798 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239950b2-3631-47d0-858b-ff76f56205ed None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:25.249040 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-239950b2-3631-47d0-858b-ff76f56205ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:25.315875 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02852a0c-62ef-4428-9998-cd2373bc4731 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:25.334979 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba00e7d-83a4-46e3-8ae4-f2ced4c305e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:25.335939 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba00e7d-83a4-46e3-8ae4-f2ced4c305e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:25.338532 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba00e7d-83a4-46e3-8ae4-f2ced4c305e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:25.340020 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba00e7d-83a4-46e3-8ae4-f2ced4c305e3 admin admin] PATH_INFO: `/v3/projects/95161229e3f14e42857866e55b39ec09` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:25.348309 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba00e7d-83a4-46e3-8ae4-f2ced4c305e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95161229e3f14e42857866e55b39ec09)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:25.353494 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba00e7d-83a4-46e3-8ae4-f2ced4c305e3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:25.445893 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:25.446756 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239950b2-3631-47d0-858b-ff76f56205ed None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:25.472382 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adca7e5-dcc2-4009-bb89-818a8aa7e023 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:25.473364 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adca7e5-dcc2-4009-bb89-818a8aa7e023 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:25.473364 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adca7e5-dcc2-4009-bb89-818a8aa7e023 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:25.473606 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adca7e5-dcc2-4009-bb89-818a8aa7e023 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:25.490824 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adca7e5-dcc2-4009-bb89-818a8aa7e023 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:25.492270 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adca7e5-dcc2-4009-bb89-818a8aa7e023 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:25.519250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 840/1671] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:21:25.543751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adca7e5-dcc2-4009-bb89-818a8aa7e023 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:25.547639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0adca7e5-dcc2-4009-bb89-818a8aa7e023 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:25.556201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2ba00e7d-83a4-46e3-8ae4-f2ced4c305e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95161229e3f14e42857866e55b39ec09', 'request_id': 'req-2ba00e7d-83a4-46e3-8ae4-f2ced4c305e3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:25.575391 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2ba00e7d-83a4-46e3-8ae4-f2ced4c305e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95161229e3f14e42857866e55b39ec09', 'request_id': 'req-2ba00e7d-83a4-46e3-8ae4-f2ced4c305e3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:25.581498 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adca7e5-dcc2-4009-bb89-818a8aa7e023 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:25.590439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2ba00e7d-83a4-46e3-8ae4-f2ced4c305e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95161229e3f14e42857866e55b39ec09', 'request_id': 'req-2ba00e7d-83a4-46e3-8ae4-f2ced4c305e3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:25.626814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2ba00e7d-83a4-46e3-8ae4-f2ced4c305e3 admin admin] The token cache is being invalidate because project 95161229e3f14e42857866e55b39ec09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:21:25.632614 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2ba00e7d-83a4-46e3-8ae4-f2ced4c305e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:25.634949 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2ba00e7d-83a4-46e3-8ae4-f2ced4c305e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:25.642274 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 832/1672] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:21:25 2022] DELETE /identity/v3/projects/95161229e3f14e42857866e55b39ec09 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:25.645405 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c6f3f-1bfb-40df-9106-cd07daa574b6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:25.645696 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c6f3f-1bfb-40df-9106-cd07daa574b6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:25.645976 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c6f3f-1bfb-40df-9106-cd07daa574b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:25.664050 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:25.704710 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e73c6f3f-1bfb-40df-9106-cd07daa574b6 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:25.709113 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e73c6f3f-1bfb-40df-9106-cd07daa574b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:25.759064 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c44103-9fe8-411b-baf8-730ae85ee1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:25.760250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c44103-9fe8-411b-baf8-730ae85ee1f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:25.760549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c44103-9fe8-411b-baf8-730ae85ee1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:25.760828 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c44103-9fe8-411b-baf8-730ae85ee1f6 admin admin] PATH_INFO: `/v3/projects/b284a7c77bd04bb38bc9dd74c3c6a22b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:25.766590 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c44103-9fe8-411b-baf8-730ae85ee1f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b284a7c77bd04bb38bc9dd74c3c6a22b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:25.775344 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c44103-9fe8-411b-baf8-730ae85ee1f6 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:25.828852 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 833/1673] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:25.838671 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb3bcf-debb-49c0-9261-6888f9cbcf1f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:25.839955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb3bcf-debb-49c0-9261-6888f9cbcf1f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:25.840251 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb3bcf-debb-49c0-9261-6888f9cbcf1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:25.881410 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-39cb3bcf-debb-49c0-9261-6888f9cbcf1f None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:25.883615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-39cb3bcf-debb-49c0-9261-6888f9cbcf1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:25.895781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f1c44103-9fe8-411b-baf8-730ae85ee1f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b284a7c77bd04bb38bc9dd74c3c6a22b', 'request_id': 'req-f1c44103-9fe8-411b-baf8-730ae85ee1f6'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:25.910851 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f1c44103-9fe8-411b-baf8-730ae85ee1f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b284a7c77bd04bb38bc9dd74c3c6a22b', 'request_id': 'req-f1c44103-9fe8-411b-baf8-730ae85ee1f6'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:25.931151 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f1c44103-9fe8-411b-baf8-730ae85ee1f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b284a7c77bd04bb38bc9dd74c3c6a22b', 'request_id': 'req-f1c44103-9fe8-411b-baf8-730ae85ee1f6'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:25.960192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f1c44103-9fe8-411b-baf8-730ae85ee1f6 admin admin] The token cache is being invalidate because project b284a7c77bd04bb38bc9dd74c3c6a22b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:21:25.960596 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f1c44103-9fe8-411b-baf8-730ae85ee1f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:25.962405 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f1c44103-9fe8-411b-baf8-730ae85ee1f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:25.967405 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 841/1674] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:21:25 2022] DELETE /identity/v3/projects/b284a7c77bd04bb38bc9dd74c3c6a22b => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:25.972989 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49abe7-586f-48d0-aaae-e2cfd0e5f190 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:25.973388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49abe7-586f-48d0-aaae-e2cfd0e5f190 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:25.973779 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49abe7-586f-48d0-aaae-e2cfd0e5f190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:26.027413 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 834/1675] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:26.030762 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb3bcf-debb-49c0-9261-6888f9cbcf1f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:26.038491 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6c49abe7-586f-48d0-aaae-e2cfd0e5f190 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:26.038491 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6c49abe7-586f-48d0-aaae-e2cfd0e5f190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:26.053804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb3bcf-debb-49c0-9261-6888f9cbcf1f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:26.065110 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd973c-1b32-446b-9dab-7496697f440d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:26.066282 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd973c-1b32-446b-9dab-7496697f440d admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:26.067077 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd973c-1b32-446b-9dab-7496697f440d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:26.067596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd973c-1b32-446b-9dab-7496697f440d admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:26.068792 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd973c-1b32-446b-9dab-7496697f440d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:26.069281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd973c-1b32-446b-9dab-7496697f440d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:26.071537 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd973c-1b32-446b-9dab-7496697f440d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:26.078415 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 842/1676] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:26.080403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad90aa-d32a-4cf0-8910-ffa9991a2145 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:26.080766 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad90aa-d32a-4cf0-8910-ffa9991a2145 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:26.081115 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad90aa-d32a-4cf0-8910-ffa9991a2145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:26.088304 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 835/1677] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:26.094169 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd973c-1b32-446b-9dab-7496697f440d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:26.115800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd973c-1b32-446b-9dab-7496697f440d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:26.120648 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-38ad90aa-d32a-4cf0-8910-ffa9991a2145 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:26.121897 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c1e7d-f42e-47b0-981b-84aee7c5e6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:26.122585 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c1e7d-f42e-47b0-981b-84aee7c5e6ca admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:26.123009 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c1e7d-f42e-47b0-981b-84aee7c5e6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:26.123353 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c1e7d-f42e-47b0-981b-84aee7c5e6ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:26.125182 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-38ad90aa-d32a-4cf0-8910-ffa9991a2145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:26.125858 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c1e7d-f42e-47b0-981b-84aee7c5e6ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:26.127972 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c1e7d-f42e-47b0-981b-84aee7c5e6ca admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:26.160823 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 843/1678] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:21:26.168312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 836/1679] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:26 2022] POST /identity/v3/projects => generated 405 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:26.168631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad90aa-d32a-4cf0-8910-ffa9991a2145 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:26.169678 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c1e7d-f42e-47b0-981b-84aee7c5e6ca admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:26.196310 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad90aa-d32a-4cf0-8910-ffa9991a2145 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:26.226985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:26.315944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c1e7d-f42e-47b0-981b-84aee7c5e6ca admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:26.326851 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114c2fd-c880-492d-a083-b7994d066488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:26.330853 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114c2fd-c880-492d-a083-b7994d066488 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:26.331445 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114c2fd-c880-492d-a083-b7994d066488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:26.331843 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114c2fd-c880-492d-a083-b7994d066488 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:26.333155 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1114c2fd-c880-492d-a083-b7994d066488 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:26.338941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1114c2fd-c880-492d-a083-b7994d066488 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:26.349000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc4384-105f-4686-af55-93de7a959fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:26.350930 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc4384-105f-4686-af55-93de7a959fcc admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:26.351502 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc4384-105f-4686-af55-93de7a959fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:26.352941 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc4384-105f-4686-af55-93de7a959fcc admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:26.355795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc4384-105f-4686-af55-93de7a959fcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:26.356117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc4384-105f-4686-af55-93de7a959fcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:26.360658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc4384-105f-4686-af55-93de7a959fcc admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:26.392118 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 837/1680] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:26 2022] POST /identity/v3/users => generated 303 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:26.395594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114c2fd-c880-492d-a083-b7994d066488 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:26.408560 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 844/1681] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:26.417005 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc4384-105f-4686-af55-93de7a959fcc admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:26.425472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114c2fd-c880-492d-a083-b7994d066488 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:26.439892 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15445c1-42ee-4bf3-969f-7810a0e19367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:26.440884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15445c1-42ee-4bf3-969f-7810a0e19367 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:26.441230 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15445c1-42ee-4bf3-969f-7810a0e19367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:26.441621 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15445c1-42ee-4bf3-969f-7810a0e19367 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:26.442478 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15445c1-42ee-4bf3-969f-7810a0e19367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:26.444653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15445c1-42ee-4bf3-969f-7810a0e19367 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:26.449251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc4384-105f-4686-af55-93de7a959fcc admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:26.465366 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb62a01-a16f-4e45-858e-1d604e910643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:26.469385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 838/1682] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:26.471965 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb62a01-a16f-4e45-858e-1d604e910643 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:26.472314 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb62a01-a16f-4e45-858e-1d604e910643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:26.472602 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb62a01-a16f-4e45-858e-1d604e910643 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:26.473341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb62a01-a16f-4e45-858e-1d604e910643 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:26.475530 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb62a01-a16f-4e45-858e-1d604e910643 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:26.475646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15445c1-42ee-4bf3-969f-7810a0e19367 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:26.512747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15445c1-42ee-4bf3-969f-7810a0e19367 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:26.523057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 845/1683] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:26 2022] POST /identity/v3/projects => generated 367 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:26.524428 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb62a01-a16f-4e45-858e-1d604e910643 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:26.526384 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8be060-f0a8-4e18-9151-c50e0eb60334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:26.527081 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8be060-f0a8-4e18-9151-c50e0eb60334 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:26.527397 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8be060-f0a8-4e18-9151-c50e0eb60334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:26.527676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8be060-f0a8-4e18-9151-c50e0eb60334 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:26.528407 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8be060-f0a8-4e18-9151-c50e0eb60334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:26.531028 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8be060-f0a8-4e18-9151-c50e0eb60334 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:26.555173 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 839/1684] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:26.559544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8be060-f0a8-4e18-9151-c50e0eb60334 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:26.572755 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:26.583820 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8be060-f0a8-4e18-9151-c50e0eb60334 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:26.618911 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb62a01-a16f-4e45-858e-1d604e910643 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:26.625222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc765bb9-f92d-4fe8-96da-41fbd3dc962e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:26.626383 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc765bb9-f92d-4fe8-96da-41fbd3dc962e admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:26.626969 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc765bb9-f92d-4fe8-96da-41fbd3dc962e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:26.627593 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc765bb9-f92d-4fe8-96da-41fbd3dc962e admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:26.628594 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc765bb9-f92d-4fe8-96da-41fbd3dc962e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:26.630942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc765bb9-f92d-4fe8-96da-41fbd3dc962e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:26.676012 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 846/1685] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:26 2022] POST /identity/v3/users => generated 284 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:26.677221 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc765bb9-f92d-4fe8-96da-41fbd3dc962e admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:26.691686 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc765bb9-f92d-4fe8-96da-41fbd3dc962e admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:26.700992 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ad962-c396-40da-9e43-df06cdaea9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:26.702998 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ad962-c396-40da-9e43-df06cdaea9bf admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:26.703408 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ad962-c396-40da-9e43-df06cdaea9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:26.703769 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ad962-c396-40da-9e43-df06cdaea9bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:26.704966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900ad962-c396-40da-9e43-df06cdaea9bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:26.706744 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900ad962-c396-40da-9e43-df06cdaea9bf admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:26.724085 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 847/1686] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:26.726627 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ad962-c396-40da-9e43-df06cdaea9bf admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:26.739555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27dd8bb-18ac-42e1-a8e1-bc4ee08ecd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:26.740047 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ad962-c396-40da-9e43-df06cdaea9bf admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:26.740957 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27dd8bb-18ac-42e1-a8e1-bc4ee08ecd13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:26.741488 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27dd8bb-18ac-42e1-a8e1-bc4ee08ecd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:26.741932 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27dd8bb-18ac-42e1-a8e1-bc4ee08ecd13 admin admin] PATH_INFO: `/v3/projects/152caa2f11af4b8ebb87b6616ae9c49b/users/a7a4a3bed25f4d1a941b3e6fea6bd654/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:26.746913 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aebbeb5-0b34-4927-952b-100532bb1ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:26.748403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebbeb5-0b34-4927-952b-100532bb1ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:26.748403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebbeb5-0b34-4927-952b-100532bb1ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:26.748532 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebbeb5-0b34-4927-952b-100532bb1ad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:26.749596 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aebbeb5-0b34-4927-952b-100532bb1ad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:26.751380 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aebbeb5-0b34-4927-952b-100532bb1ad7 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:26.769322 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 848/1687] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:26.770431 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aebbeb5-0b34-4927-952b-100532bb1ad7 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:26.777295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27dd8bb-18ac-42e1-a8e1-bc4ee08ecd13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=152caa2f11af4b8ebb87b6616ae9c49b, user_id=a7a4a3bed25f4d1a941b3e6fea6bd654, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:26.781064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27dd8bb-18ac-42e1-a8e1-bc4ee08ecd13 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:26.783481 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aebbeb5-0b34-4927-952b-100532bb1ad7 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:26.789149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac4283-0238-4ae4-8ba9-ccedb0e873b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:26.790705 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac4283-0238-4ae4-8ba9-ccedb0e873b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:26.791127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac4283-0238-4ae4-8ba9-ccedb0e873b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:26.791429 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac4283-0238-4ae4-8ba9-ccedb0e873b7 admin admin] PATH_INFO: `/v3/projects/e564bcf94f444f2fbda896ebf85b6593/users/dc7a2cbd571e45d2bc7791cc62f7c124/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:26.825548 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac4283-0238-4ae4-8ba9-ccedb0e873b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e564bcf94f444f2fbda896ebf85b6593, user_id=dc7a2cbd571e45d2bc7791cc62f7c124, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:26.831330 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac4283-0238-4ae4-8ba9-ccedb0e873b7 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:26.848395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 840/1688] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:26 2022] PUT /identity/v3/projects/152caa2f11af4b8ebb87b6616ae9c49b/users/a7a4a3bed25f4d1a941b3e6fea6bd654/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:26.851471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79555156-5948-48db-8f76-ecc3c010ca01 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:26.851820 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79555156-5948-48db-8f76-ecc3c010ca01 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:26.852194 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79555156-5948-48db-8f76-ecc3c010ca01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:26.869854 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 849/1689] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:26 2022] PUT /identity/v3/projects/e564bcf94f444f2fbda896ebf85b6593/users/dc7a2cbd571e45d2bc7791cc62f7c124/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:26.883633 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db97bf3-8c0a-43ea-8115-f044580430f2 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:26.884356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db97bf3-8c0a-43ea-8115-f044580430f2 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:26.884844 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db97bf3-8c0a-43ea-8115-f044580430f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:26.932755 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-79555156-5948-48db-8f76-ecc3c010ca01 None None] MFA Rules not processed for user `a7a4a3bed25f4d1a941b3e6fea6bd654`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:26.934434 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-79555156-5948-48db-8f76-ecc3c010ca01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:26.949784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-3db97bf3-8c0a-43ea-8115-f044580430f2 None None] MFA Rules not processed for user `dc7a2cbd571e45d2bc7791cc62f7c124`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:26.951640 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3db97bf3-8c0a-43ea-8115-f044580430f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:27.164798 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 850/1690] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:26 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:27.166241 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db97bf3-8c0a-43ea-8115-f044580430f2 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:27.179880 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:27.204101 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 841/1691] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:26 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:27.205605 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79555156-5948-48db-8f76-ecc3c010ca01 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:27.349083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db97bf3-8c0a-43ea-8115-f044580430f2 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:27.358207 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79555156-5948-48db-8f76-ecc3c010ca01 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:27.364027 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ba8a7-f263-4b41-87e3-21d1eb2d4cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:27.365250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4fe8f8-1615-4f67-87a8-59e4bcdba52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:27.365911 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ba8a7-f263-4b41-87e3-21d1eb2d4cb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:27.366217 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ba8a7-f263-4b41-87e3-21d1eb2d4cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:27.366545 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ba8a7-f263-4b41-87e3-21d1eb2d4cb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:27.366824 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fe8f8-1615-4f67-87a8-59e4bcdba52d admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:27.367369 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fe8f8-1615-4f67-87a8-59e4bcdba52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:27.367932 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fe8f8-1615-4f67-87a8-59e4bcdba52d admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:27.368188 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ba8a7-f263-4b41-87e3-21d1eb2d4cb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:27.369099 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4fe8f8-1615-4f67-87a8-59e4bcdba52d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:27.371823 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4fe8f8-1615-4f67-87a8-59e4bcdba52d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:27.374861 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ba8a7-f263-4b41-87e3-21d1eb2d4cb8 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:27.384958 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:27.412916 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 842/1692] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:27 2022] POST /identity/v3/projects => generated 405 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:27.414188 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4fe8f8-1615-4f67-87a8-59e4bcdba52d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:27.440303 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 851/1693] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:27 2022] POST /identity/v3/projects => generated 365 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:21:27.444020 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ba8a7-f263-4b41-87e3-21d1eb2d4cb8 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:27.625228 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4fe8f8-1615-4f67-87a8-59e4bcdba52d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:27.626004 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ba8a7-f263-4b41-87e3-21d1eb2d4cb8 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:27.635678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db1ac5-1c7d-4b0c-8d66-6195aac5eb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:27.636315 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac9cb9-e8ce-4329-a7f7-e41c0071e283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:27.637879 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db1ac5-1c7d-4b0c-8d66-6195aac5eb14 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:27.638157 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db1ac5-1c7d-4b0c-8d66-6195aac5eb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:27.638436 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db1ac5-1c7d-4b0c-8d66-6195aac5eb14 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:27.638820 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac9cb9-e8ce-4329-a7f7-e41c0071e283 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:27.639367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac9cb9-e8ce-4329-a7f7-e41c0071e283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:27.640549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db1ac5-1c7d-4b0c-8d66-6195aac5eb14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:27.640978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac9cb9-e8ce-4329-a7f7-e41c0071e283 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:27.641804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac9cb9-e8ce-4329-a7f7-e41c0071e283 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:27.645128 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db1ac5-1c7d-4b0c-8d66-6195aac5eb14 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:27.648793 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac9cb9-e8ce-4329-a7f7-e41c0071e283 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:27.711053 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 843/1694] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:27 2022] POST /identity/v3/users => generated 309 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:27.715561 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 852/1695] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:27 2022] POST /identity/v3/users => generated 289 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:27.715910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac9cb9-e8ce-4329-a7f7-e41c0071e283 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:27.718978 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db1ac5-1c7d-4b0c-8d66-6195aac5eb14 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:27.725536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4dac9cb9-e8ce-4329-a7f7-e41c0071e283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:27.743392 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db1ac5-1c7d-4b0c-8d66-6195aac5eb14 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:27.751705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac9cb9-e8ce-4329-a7f7-e41c0071e283 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:27.754665 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e018c6-26e2-4e02-8aec-0e689edd5ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:27.755394 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e018c6-26e2-4e02-8aec-0e689edd5ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:27.756125 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e018c6-26e2-4e02-8aec-0e689edd5ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:27.756440 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e018c6-26e2-4e02-8aec-0e689edd5ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:27.757189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e018c6-26e2-4e02-8aec-0e689edd5ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:27.758805 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e018c6-26e2-4e02-8aec-0e689edd5ac6 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:27.773710 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fea2a2-8c1b-4bf3-aeb6-776d51057ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:27.774607 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fea2a2-8c1b-4bf3-aeb6-776d51057ca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:27.774889 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fea2a2-8c1b-4bf3-aeb6-776d51057ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:27.775164 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fea2a2-8c1b-4bf3-aeb6-776d51057ca8 admin admin] PATH_INFO: `/v3/projects/9e5535c0d612453ab394b0d90b0e6e02` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:27.778212 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fea2a2-8c1b-4bf3-aeb6-776d51057ca8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e5535c0d612453ab394b0d90b0e6e02)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:27.783609 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fea2a2-8c1b-4bf3-aeb6-776d51057ca8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:27.800707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 853/1696] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:27.801611 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e018c6-26e2-4e02-8aec-0e689edd5ac6 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:27.828459 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e018c6-26e2-4e02-8aec-0e689edd5ac6 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:27.842566 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567713b3-5dfb-481b-920d-74e93f0dc0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:27.844125 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567713b3-5dfb-481b-920d-74e93f0dc0e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:27.846397 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567713b3-5dfb-481b-920d-74e93f0dc0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:27.846823 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567713b3-5dfb-481b-920d-74e93f0dc0e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:27.850147 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567713b3-5dfb-481b-920d-74e93f0dc0e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:27.852598 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567713b3-5dfb-481b-920d-74e93f0dc0e1 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:27.872571 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 854/1697] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:27.876156 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfeb8d4-cdd8-4dc7-ac0e-33d6a4a7afe4 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:27.876482 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfeb8d4-cdd8-4dc7-ac0e-33d6a4a7afe4 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:27.876764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfeb8d4-cdd8-4dc7-ac0e-33d6a4a7afe4 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:27.879580 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 855/1698] 10.222.0.81 () {54 vars in 921 bytes} [Tue Jun 7 14:21:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:27.880928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfeb8d4-cdd8-4dc7-ac0e-33d6a4a7afe4 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:27.893319 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54fea2a2-8c1b-4bf3-aeb6-776d51057ca8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e5535c0d612453ab394b0d90b0e6e02', 'request_id': 'req-54fea2a2-8c1b-4bf3-aeb6-776d51057ca8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:27.904394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54fea2a2-8c1b-4bf3-aeb6-776d51057ca8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e5535c0d612453ab394b0d90b0e6e02', 'request_id': 'req-54fea2a2-8c1b-4bf3-aeb6-776d51057ca8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:27.915480 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54fea2a2-8c1b-4bf3-aeb6-776d51057ca8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e5535c0d612453ab394b0d90b0e6e02', 'request_id': 'req-54fea2a2-8c1b-4bf3-aeb6-776d51057ca8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:27.918590 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:27.940049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54fea2a2-8c1b-4bf3-aeb6-776d51057ca8 admin admin] The token cache is being invalidate because project 9e5535c0d612453ab394b0d90b0e6e02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:21:27.940382 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54fea2a2-8c1b-4bf3-aeb6-776d51057ca8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:27.943930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54fea2a2-8c1b-4bf3-aeb6-776d51057ca8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:27.948335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 844/1699] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:21:27 2022] DELETE /identity/v3/projects/9e5535c0d612453ab394b0d90b0e6e02 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:27.950072 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fea2a2-8c1b-4bf3-aeb6-776d51057ca8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:27.962046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-54fea2a2-8c1b-4bf3-aeb6-776d51057ca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:28.070807 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfeb8d4-cdd8-4dc7-ac0e-33d6a4a7afe4 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:28.081787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61062b0c-d02b-4581-9bfc-901bf2ed4436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:28.083649 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61062b0c-d02b-4581-9bfc-901bf2ed4436 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:28.084241 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61062b0c-d02b-4581-9bfc-901bf2ed4436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:28.085255 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61062b0c-d02b-4581-9bfc-901bf2ed4436 admin admin] PATH_INFO: `/v3/projects/49f7cc86181f4d8aa35593d8f6339e42/users/881c17d6099b4c70a705a096b273025f/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:28.113821 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fea2a2-8c1b-4bf3-aeb6-776d51057ca8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:28.118379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61062b0c-d02b-4581-9bfc-901bf2ed4436 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49f7cc86181f4d8aa35593d8f6339e42, user_id=881c17d6099b4c70a705a096b273025f, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:28.123410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61062b0c-d02b-4581-9bfc-901bf2ed4436 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:28.131118 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de00521-649a-453f-bfe6-d58fb29e2bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:28.132701 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de00521-649a-453f-bfe6-d58fb29e2bcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:28.133183 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de00521-649a-453f-bfe6-d58fb29e2bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:28.134235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de00521-649a-453f-bfe6-d58fb29e2bcf admin admin] PATH_INFO: `/v3/projects/29c2010db6434f87a9111c40512bc4e5/users/f09d7af261fe4e94a3f8e87a91180677/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:28.155787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 856/1700] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:27 2022] PUT /identity/v3/projects/49f7cc86181f4d8aa35593d8f6339e42/users/881c17d6099b4c70a705a096b273025f/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:28.159492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7b8ce-b8d5-4624-bad8-05422c075aeb None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:28.159892 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7b8ce-b8d5-4624-bad8-05422c075aeb None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:28.160269 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7b8ce-b8d5-4624-bad8-05422c075aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:28.172618 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de00521-649a-453f-bfe6-d58fb29e2bcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=29c2010db6434f87a9111c40512bc4e5, user_id=f09d7af261fe4e94a3f8e87a91180677, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:28.185416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de00521-649a-453f-bfe6-d58fb29e2bcf admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:28.205393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-e7e7b8ce-b8d5-4624-bad8-05422c075aeb None None] MFA Rules not processed for user `ab73cf3febbf4c768cd0116306f16c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:28.208425 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e7e7b8ce-b8d5-4624-bad8-05422c075aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:28.221453 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 845/1701] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:27 2022] PUT /identity/v3/projects/29c2010db6434f87a9111c40512bc4e5/users/f09d7af261fe4e94a3f8e87a91180677/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:28.221453 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de00521-649a-453f-bfe6-d58fb29e2bcf admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:28.223539 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9de00521-649a-453f-bfe6-d58fb29e2bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:28.302686 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:28.393330 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 857/1702] 10.222.0.81 () {58 vars in 1018 bytes} [Tue Jun 7 14:21:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:28.394399 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7b8ce-b8d5-4624-bad8-05422c075aeb None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:28.404056 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e7e7b8ce-b8d5-4624-bad8-05422c075aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:28.412992 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de00521-649a-453f-bfe6-d58fb29e2bcf admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:28.425963 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58760d-d9f6-4059-81e5-f177cb63ac92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:28.427131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58760d-d9f6-4059-81e5-f177cb63ac92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:28.427550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58760d-d9f6-4059-81e5-f177cb63ac92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:28.428122 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58760d-d9f6-4059-81e5-f177cb63ac92 admin admin] PATH_INFO: `/v3/users/f04282fce71d49e29fdff413ba73f7a2` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:28.431504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb58760d-d9f6-4059-81e5-f177cb63ac92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f04282fce71d49e29fdff413ba73f7a2)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:28.439649 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb58760d-d9f6-4059-81e5-f177cb63ac92 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:28.442986 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7b8ce-b8d5-4624-bad8-05422c075aeb None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:28.455863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a5a18-1716-4441-b852-4e77c1309779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:28.456750 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a5a18-1716-4441-b852-4e77c1309779 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:28.458940 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a5a18-1716-4441-b852-4e77c1309779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:28.459712 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a5a18-1716-4441-b852-4e77c1309779 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:28.464882 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a5a18-1716-4441-b852-4e77c1309779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:28.469848 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a5a18-1716-4441-b852-4e77c1309779 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:28.505013 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 858/1703] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:28.506613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a5a18-1716-4441-b852-4e77c1309779 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:28.535979 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a5a18-1716-4441-b852-4e77c1309779 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:28.551795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692fa43-3cb0-4739-9cc7-dd9f08443a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:28.552540 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692fa43-3cb0-4739-9cc7-dd9f08443a2a admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:28.554122 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692fa43-3cb0-4739-9cc7-dd9f08443a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:28.554403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692fa43-3cb0-4739-9cc7-dd9f08443a2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:28.555161 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692fa43-3cb0-4739-9cc7-dd9f08443a2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:28.556819 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692fa43-3cb0-4739-9cc7-dd9f08443a2a admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:28.576452 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 859/1704] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:28.577111 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cb58760d-d9f6-4059-81e5-f177cb63ac92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f04282fce71d49e29fdff413ba73f7a2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:28.580347 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cb58760d-d9f6-4059-81e5-f177cb63ac92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f04282fce71d49e29fdff413ba73f7a2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:28.580564 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692fa43-3cb0-4739-9cc7-dd9f08443a2a admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:28.582298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cb58760d-d9f6-4059-81e5-f177cb63ac92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f04282fce71d49e29fdff413ba73f7a2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:28.586476 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a692fa43-3cb0-4739-9cc7-dd9f08443a2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:28.611817 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cb58760d-d9f6-4059-81e5-f177cb63ac92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f04282fce71d49e29fdff413ba73f7a2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:28.633079 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692fa43-3cb0-4739-9cc7-dd9f08443a2a admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:28.648297 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb0a663-1571-4ee8-a021-84beb8710223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:28.649506 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0a663-1571-4ee8-a021-84beb8710223 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:28.650044 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0a663-1571-4ee8-a021-84beb8710223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:28.650522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0a663-1571-4ee8-a021-84beb8710223 admin admin] PATH_INFO: `/v3/projects/49f7cc86181f4d8aa35593d8f6339e42/users/881c17d6099b4c70a705a096b273025f/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:28.653136 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb0a663-1571-4ee8-a021-84beb8710223 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49f7cc86181f4d8aa35593d8f6339e42, user_id=881c17d6099b4c70a705a096b273025f, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:28.659891 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb0a663-1571-4ee8-a021-84beb8710223 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:28.661545 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cb58760d-d9f6-4059-81e5-f177cb63ac92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f04282fce71d49e29fdff413ba73f7a2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:28.683880 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 860/1705] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:28 2022] PUT /identity/v3/projects/49f7cc86181f4d8aa35593d8f6339e42/users/881c17d6099b4c70a705a096b273025f/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:28.685935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e87c4-17a4-4e4c-b099-5c4f193d14f9 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:28.686975 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e87c4-17a4-4e4c-b099-5c4f193d14f9 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:28.687535 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e87c4-17a4-4e4c-b099-5c4f193d14f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:28.699709 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:28.714237 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 846/1706] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:21:28 2022] DELETE /identity/v3/users/f04282fce71d49e29fdff413ba73f7a2 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:28.715304 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58760d-d9f6-4059-81e5-f177cb63ac92 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:28.718732 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cb58760d-d9f6-4059-81e5-f177cb63ac92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:28.727390 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-9b2e87c4-17a4-4e4c-b099-5c4f193d14f9 None None] MFA Rules not processed for user `d00b8bc311234e6da47b365ac8a7183a`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:28.729382 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9b2e87c4-17a4-4e4c-b099-5c4f193d14f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:28.879730 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58760d-d9f6-4059-81e5-f177cb63ac92 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:28.881509 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 861/1707] 10.222.0.81 () {58 vars in 1009 bytes} [Tue Jun 7 14:21:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:28.882552 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e87c4-17a4-4e4c-b099-5c4f193d14f9 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:28.885125 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9b2e87c4-17a4-4e4c-b099-5c4f193d14f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:28.903098 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eafff8-7d60-4b10-bf3b-909b8b793c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:28.904883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eafff8-7d60-4b10-bf3b-909b8b793c4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:28.905411 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eafff8-7d60-4b10-bf3b-909b8b793c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:28.906190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eafff8-7d60-4b10-bf3b-909b8b793c4d admin admin] PATH_INFO: `/v3/projects/29c2010db6434f87a9111c40512bc4e5/users/f09d7af261fe4e94a3f8e87a91180677/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:28.944584 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eafff8-7d60-4b10-bf3b-909b8b793c4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=29c2010db6434f87a9111c40512bc4e5, user_id=f09d7af261fe4e94a3f8e87a91180677, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:28.947303 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eafff8-7d60-4b10-bf3b-909b8b793c4d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:28.972487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 847/1708] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:28 2022] PUT /identity/v3/projects/29c2010db6434f87a9111c40512bc4e5/users/f09d7af261fe4e94a3f8e87a91180677/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:28.973820 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5d0a7-3b64-4f82-91bc-0a6cc29f37ca None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:28.974102 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5d0a7-3b64-4f82-91bc-0a6cc29f37ca None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:28.979117 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5d0a7-3b64-4f82-91bc-0a6cc29f37ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:29.018923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e87c4-17a4-4e4c-b099-5c4f193d14f9 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:29.057158 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-26a5d0a7-3b64-4f82-91bc-0a6cc29f37ca None None] MFA Rules not processed for user `881c17d6099b4c70a705a096b273025f`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:29.058851 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-26a5d0a7-3b64-4f82-91bc-0a6cc29f37ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:29.089277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:29.120981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c129f13-d0c3-41b9-9378-08555505e6d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab73cf3febbf4c768cd0116306f16c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:29.121707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c129f13-d0c3-41b9-9378-08555505e6d1 service placement] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:29.122042 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c129f13-d0c3-41b9-9378-08555505e6d1 service placement] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:29.122319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c129f13-d0c3-41b9-9378-08555505e6d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:29.127453 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9c129f13-d0c3-41b9-9378-08555505e6d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:29.144232 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c129f13-d0c3-41b9-9378-08555505e6d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:29.146013 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c129f13-d0c3-41b9-9378-08555505e6d1 service placement] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:29.165793 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 862/1709] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:29.168742 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c2ec3-34a9-4300-93a2-9d325f05a655 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:29.169090 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c2ec3-34a9-4300-93a2-9d325f05a655 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:29.169438 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c2ec3-34a9-4300-93a2-9d325f05a655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:29.214514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 848/1710] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:28 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:29.215977 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb99d0-eeb2-4578-8655-fffea6ebfc1a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:29.216284 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb99d0-eeb2-4578-8655-fffea6ebfc1a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:29.216638 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb99d0-eeb2-4578-8655-fffea6ebfc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:29.226587 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-764c2ec3-34a9-4300-93a2-9d325f05a655 None None] MFA Rules not processed for user `d20025ab12f4450da20c8e41b931b5fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:29.227619 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-764c2ec3-34a9-4300-93a2-9d325f05a655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:29.273938 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-06bb99d0-eeb2-4578-8655-fffea6ebfc1a None None] MFA Rules not processed for user `f09d7af261fe4e94a3f8e87a91180677`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:29.275477 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-06bb99d0-eeb2-4578-8655-fffea6ebfc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:29.379143 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 849/1711] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:29 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:29.380399 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb99d0-eeb2-4578-8655-fffea6ebfc1a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:29.382350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-06bb99d0-eeb2-4578-8655-fffea6ebfc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:29.386233 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 863/1712] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:29.387321 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c2ec3-34a9-4300-93a2-9d325f05a655 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:29.396670 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-764c2ec3-34a9-4300-93a2-9d325f05a655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:29.413350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:29.549449 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb99d0-eeb2-4578-8655-fffea6ebfc1a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:29.549932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c2ec3-34a9-4300-93a2-9d325f05a655 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:29.559977 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f6a0b-73c6-42bd-b836-dadb35bf7c3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:29.562201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f6a0b-73c6-42bd-b836-dadb35bf7c3b service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:29.562799 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f6a0b-73c6-42bd-b836-dadb35bf7c3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:29.563285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f6a0b-73c6-42bd-b836-dadb35bf7c3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:29.567632 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334c8f8-44ef-4ba7-ab44-c446a056b10a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da7c0f99060a44eaab743ba71f159e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:29.568402 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334c8f8-44ef-4ba7-ab44-c446a056b10a service swift] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:29.568657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334c8f8-44ef-4ba7-ab44-c446a056b10a service swift] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:29.568978 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334c8f8-44ef-4ba7-ab44-c446a056b10a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:29.585982 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f6a0b-73c6-42bd-b836-dadb35bf7c3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:29.591208 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f6a0b-73c6-42bd-b836-dadb35bf7c3b service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:29.628507 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 850/1713] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:29 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:29.629600 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfe156-57bc-4b77-b3ca-ee0d218b4122 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:29.629893 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfe156-57bc-4b77-b3ca-ee0d218b4122 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:29.630172 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfe156-57bc-4b77-b3ca-ee0d218b4122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:29.662637 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6cdfe156-57bc-4b77-b3ca-ee0d218b4122 None None] MFA Rules not processed for user `d20025ab12f4450da20c8e41b931b5fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:29.663931 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6cdfe156-57bc-4b77-b3ca-ee0d218b4122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:29.678479 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 851/1714] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:29.680413 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdfe156-57bc-4b77-b3ca-ee0d218b4122 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:29.681766 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6cdfe156-57bc-4b77-b3ca-ee0d218b4122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:29.696613 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdfe156-57bc-4b77-b3ca-ee0d218b4122 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:29.697039 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334c8f8-44ef-4ba7-ab44-c446a056b10a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:29.698701 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334c8f8-44ef-4ba7-ab44-c446a056b10a service swift] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:29.700692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2861d-c28a-4a47-92f6-f50d8e491d1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:29.701356 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2861d-c28a-4a47-92f6-f50d8e491d1d service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:29.701633 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2861d-c28a-4a47-92f6-f50d8e491d1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:29.701904 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2861d-c28a-4a47-92f6-f50d8e491d1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:29.722270 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 864/1715] 10.222.0.81 () {60 vars in 1384 bytes} [Tue Jun 7 14:21:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:21:29.723155 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334c8f8-44ef-4ba7-ab44-c446a056b10a service swift] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:29.726473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b334c8f8-44ef-4ba7-ab44-c446a056b10a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:29.728050 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2861d-c28a-4a47-92f6-f50d8e491d1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:29.729514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2861d-c28a-4a47-92f6-f50d8e491d1d service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:29.755513 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:29.771621 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 852/1716] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:29 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:29.776465 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2861d-c28a-4a47-92f6-f50d8e491d1d service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:29.779749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5de2861d-c28a-4a47-92f6-f50d8e491d1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:29.941599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334c8f8-44ef-4ba7-ab44-c446a056b10a service swift] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:29.953475 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26260bab-91b9-47fd-a9ee-654958907147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:29.956553 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26260bab-91b9-47fd-a9ee-654958907147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:29.956851 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26260bab-91b9-47fd-a9ee-654958907147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:29.957104 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26260bab-91b9-47fd-a9ee-654958907147 admin admin] PATH_INFO: `/v3/projects/19af625105d842adbe8b07a4a0f44118` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:29.962217 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26260bab-91b9-47fd-a9ee-654958907147 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19af625105d842adbe8b07a4a0f44118)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:29.966271 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26260bab-91b9-47fd-a9ee-654958907147 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:30.036115 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2861d-c28a-4a47-92f6-f50d8e491d1d service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:30.060446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3921bdf-c520-4000-ab91-1c445338bbdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:30.063471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3921bdf-c520-4000-ab91-1c445338bbdf service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:30.065548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3921bdf-c520-4000-ab91-1c445338bbdf service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:30.067355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3921bdf-c520-4000-ab91-1c445338bbdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:30.128280 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3921bdf-c520-4000-ab91-1c445338bbdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:30.129895 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3921bdf-c520-4000-ab91-1c445338bbdf service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:30.178368 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 853/1717] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:29 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:30.180652 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3921bdf-c520-4000-ab91-1c445338bbdf service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:30.188069 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-26260bab-91b9-47fd-a9ee-654958907147 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19af625105d842adbe8b07a4a0f44118', 'request_id': 'req-26260bab-91b9-47fd-a9ee-654958907147'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:30.208388 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3921bdf-c520-4000-ab91-1c445338bbdf service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:30.211717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-26260bab-91b9-47fd-a9ee-654958907147 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19af625105d842adbe8b07a4a0f44118', 'request_id': 'req-26260bab-91b9-47fd-a9ee-654958907147'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:30.245342 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-26260bab-91b9-47fd-a9ee-654958907147 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19af625105d842adbe8b07a4a0f44118', 'request_id': 'req-26260bab-91b9-47fd-a9ee-654958907147'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:30.272213 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-26260bab-91b9-47fd-a9ee-654958907147 admin admin] The token cache is being invalidate because project 19af625105d842adbe8b07a4a0f44118 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:21:30.272666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-26260bab-91b9-47fd-a9ee-654958907147 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:30.274439 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-26260bab-91b9-47fd-a9ee-654958907147 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:30.287151 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 865/1718] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:21:29 2022] DELETE /identity/v3/projects/19af625105d842adbe8b07a4a0f44118 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:30.288916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26260bab-91b9-47fd-a9ee-654958907147 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:30.292681 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-26260bab-91b9-47fd-a9ee-654958907147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:30.428000 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d03f5c-f83f-4d88-ad26-113c28a6c8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:30.429051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d03f5c-f83f-4d88-ad26-113c28a6c8dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:30.429334 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d03f5c-f83f-4d88-ad26-113c28a6c8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:30.429606 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d03f5c-f83f-4d88-ad26-113c28a6c8dc admin admin] PATH_INFO: `/v3/users/a7a4a3bed25f4d1a941b3e6fea6bd654` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:30.434393 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d03f5c-f83f-4d88-ad26-113c28a6c8dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7a4a3bed25f4d1a941b3e6fea6bd654)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:30.442232 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d03f5c-f83f-4d88-ad26-113c28a6c8dc admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:30.475138 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26260bab-91b9-47fd-a9ee-654958907147 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:30.481187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:30.487134 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b845b1f4-914d-43eb-8119-e502082a6a5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:30.488031 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845b1f4-914d-43eb-8119-e502082a6a5d service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:30.488309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845b1f4-914d-43eb-8119-e502082a6a5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:30.488576 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845b1f4-914d-43eb-8119-e502082a6a5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:30.491362 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b845b1f4-914d-43eb-8119-e502082a6a5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:30.554172 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a6d03f5c-f83f-4d88-ad26-113c28a6c8dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7a4a3bed25f4d1a941b3e6fea6bd654'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:30.556454 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a6d03f5c-f83f-4d88-ad26-113c28a6c8dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7a4a3bed25f4d1a941b3e6fea6bd654'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:30.559295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a6d03f5c-f83f-4d88-ad26-113c28a6c8dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7a4a3bed25f4d1a941b3e6fea6bd654'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:30.598790 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a6d03f5c-f83f-4d88-ad26-113c28a6c8dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7a4a3bed25f4d1a941b3e6fea6bd654'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:30.641360 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a6d03f5c-f83f-4d88-ad26-113c28a6c8dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7a4a3bed25f4d1a941b3e6fea6bd654'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:30.664196 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:30.690420 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 854/1719] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:21:30 2022] DELETE /identity/v3/users/a7a4a3bed25f4d1a941b3e6fea6bd654 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:30.694776 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d03f5c-f83f-4d88-ad26-113c28a6c8dc admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:30.697156 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b845b1f4-914d-43eb-8119-e502082a6a5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:30.698178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a6d03f5c-f83f-4d88-ad26-113c28a6c8dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:30.704146 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b845b1f4-914d-43eb-8119-e502082a6a5d service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:30.732929 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 866/1720] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:30 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:30.734805 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d9b8b-627d-4dc0-a080-189595408e33 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:30.735142 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d9b8b-627d-4dc0-a080-189595408e33 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:30.738273 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d9b8b-627d-4dc0-a080-189595408e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:30.803030 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-bd6d9b8b-627d-4dc0-a080-189595408e33 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:30.804297 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-bd6d9b8b-627d-4dc0-a080-189595408e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:30.942053 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d03f5c-f83f-4d88-ad26-113c28a6c8dc admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:30.963261 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30800be-2411-4e01-b4f1-68a627c0a59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:30.966198 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30800be-2411-4e01-b4f1-68a627c0a59c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:30.967595 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30800be-2411-4e01-b4f1-68a627c0a59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:30.971036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30800be-2411-4e01-b4f1-68a627c0a59c admin admin] PATH_INFO: `/v3/users/8e440699aceb4636bb68e796e811e758` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:30.977191 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30800be-2411-4e01-b4f1-68a627c0a59c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e440699aceb4636bb68e796e811e758)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:30.979986 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30800be-2411-4e01-b4f1-68a627c0a59c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:30.988235 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 867/1721] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:30.989770 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead199b-6525-46d3-afea-1e02fe0e60b6 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:30.990066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead199b-6525-46d3-afea-1e02fe0e60b6 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:30.990349 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead199b-6525-46d3-afea-1e02fe0e60b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:31.040315 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-2ead199b-6525-46d3-afea-1e02fe0e60b6 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:31.041550 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2ead199b-6525-46d3-afea-1e02fe0e60b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:31.072085 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 868/1722] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:31.073543 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56179a74-7a1f-400d-a3fb-15c7ed29c8f3 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:31.073836 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56179a74-7a1f-400d-a3fb-15c7ed29c8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:31.074115 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56179a74-7a1f-400d-a3fb-15c7ed29c8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:31.121678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-56179a74-7a1f-400d-a3fb-15c7ed29c8f3 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:31.122922 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-56179a74-7a1f-400d-a3fb-15c7ed29c8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:31.178638 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e30800be-2411-4e01-b4f1-68a627c0a59c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e440699aceb4636bb68e796e811e758'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:31.180058 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e30800be-2411-4e01-b4f1-68a627c0a59c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e440699aceb4636bb68e796e811e758'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:31.181872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 869/1723] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:31.185153 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e30800be-2411-4e01-b4f1-68a627c0a59c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e440699aceb4636bb68e796e811e758'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:31.187790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56179a74-7a1f-400d-a3fb-15c7ed29c8f3 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:31.215907 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56179a74-7a1f-400d-a3fb-15c7ed29c8f3 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:31.217212 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e30800be-2411-4e01-b4f1-68a627c0a59c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e440699aceb4636bb68e796e811e758'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:31.241775 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3a898-3edc-49a1-8d11-2c8e063a9f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:31.242465 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3a898-3edc-49a1-8d11-2c8e063a9f88 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:31.242749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3a898-3edc-49a1-8d11-2c8e063a9f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:31.243024 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3a898-3edc-49a1-8d11-2c8e063a9f88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:31.243801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b3a898-3edc-49a1-8d11-2c8e063a9f88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:31.244284 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b3a898-3edc-49a1-8d11-2c8e063a9f88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:31.247046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b3a898-3edc-49a1-8d11-2c8e063a9f88 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:31.260387 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e30800be-2411-4e01-b4f1-68a627c0a59c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e440699aceb4636bb68e796e811e758'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:31.300178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 855/1724] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:21:30 2022] DELETE /identity/v3/users/8e440699aceb4636bb68e796e811e758 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:31.301492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 870/1725] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:31.309884 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3a898-3edc-49a1-8d11-2c8e063a9f88 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:31.310489 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bf195-bf0c-4432-9b1c-8ce8465b7252 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:31.310606 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bf195-bf0c-4432-9b1c-8ce8465b7252 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:31.312191 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bf195-bf0c-4432-9b1c-8ce8465b7252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:31.318903 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-24b3a898-3edc-49a1-8d11-2c8e063a9f88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:31.365459 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3a898-3edc-49a1-8d11-2c8e063a9f88 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:31.383516 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f45bf195-bf0c-4432-9b1c-8ce8465b7252 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:31.385574 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f45bf195-bf0c-4432-9b1c-8ce8465b7252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:31.408875 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:31.548039 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90edac49-d70c-4516-9dc3-d1a5fc66d50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:31.556600 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edac49-d70c-4516-9dc3-d1a5fc66d50b admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:31.556959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edac49-d70c-4516-9dc3-d1a5fc66d50b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:31.557326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edac49-d70c-4516-9dc3-d1a5fc66d50b admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:31.558665 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90edac49-d70c-4516-9dc3-d1a5fc66d50b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:31.566784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90edac49-d70c-4516-9dc3-d1a5fc66d50b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:31.681408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 856/1726] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:31.688479 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bf195-bf0c-4432-9b1c-8ce8465b7252 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:31.692927 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 871/1727] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:31 2022] POST /identity/v3/projects => generated 377 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:31.693551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f45bf195-bf0c-4432-9b1c-8ce8465b7252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:31.695321 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90edac49-d70c-4516-9dc3-d1a5fc66d50b admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:31.836041 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:31.886165 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90edac49-d70c-4516-9dc3-d1a5fc66d50b admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:31.900155 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f9463-465a-418d-9f8c-713bbfdc1a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:31.901907 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f9463-465a-418d-9f8c-713bbfdc1a27 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:31.902321 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f9463-465a-418d-9f8c-713bbfdc1a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:31.903950 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f9463-465a-418d-9f8c-713bbfdc1a27 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:31.916179 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f9463-465a-418d-9f8c-713bbfdc1a27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:31.926762 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f9463-465a-418d-9f8c-713bbfdc1a27 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:31.959149 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bf195-bf0c-4432-9b1c-8ce8465b7252 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:31.983456 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 872/1728] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:31 2022] POST /identity/v3/users => generated 289 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:31.986306 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f9463-465a-418d-9f8c-713bbfdc1a27 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:32.187340 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4420114-ff2b-406a-bf1a-489b8a4009c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:32.188048 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4420114-ff2b-406a-bf1a-489b8a4009c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:32.188349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4420114-ff2b-406a-bf1a-489b8a4009c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:32.188644 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4420114-ff2b-406a-bf1a-489b8a4009c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:32.200895 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b4420114-ff2b-406a-bf1a-489b8a4009c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:32.248288 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f9463-465a-418d-9f8c-713bbfdc1a27 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:32.249050 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4420114-ff2b-406a-bf1a-489b8a4009c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:32.250493 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4420114-ff2b-406a-bf1a-489b8a4009c4 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:32.268691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f087b61-7f18-4cee-859b-bf05458159e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:32.269392 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f087b61-7f18-4cee-859b-bf05458159e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:32.269720 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f087b61-7f18-4cee-859b-bf05458159e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:32.272685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f087b61-7f18-4cee-859b-bf05458159e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:32.290081 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 857/1729] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:32.297503 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4420114-ff2b-406a-bf1a-489b8a4009c4 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:32.315754 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f087b61-7f18-4cee-859b-bf05458159e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:32.317232 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f087b61-7f18-4cee-859b-bf05458159e8 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:32.329534 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4420114-ff2b-406a-bf1a-489b8a4009c4 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:32.349549 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e17c42-8af0-49f5-956e-dde5e7839715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:32.350248 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e17c42-8af0-49f5-956e-dde5e7839715 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:32.350561 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e17c42-8af0-49f5-956e-dde5e7839715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:32.350850 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e17c42-8af0-49f5-956e-dde5e7839715 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:32.353863 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e17c42-8af0-49f5-956e-dde5e7839715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:32.354703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 873/1730] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:32.364260 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e17c42-8af0-49f5-956e-dde5e7839715 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:32.414632 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 858/1731] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:32.416705 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f087b61-7f18-4cee-859b-bf05458159e8 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:32.431771 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f087b61-7f18-4cee-859b-bf05458159e8 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:32.457516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65dcace-c66e-4922-8c6c-8935c717db8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:32.458366 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dcace-c66e-4922-8c6c-8935c717db8e admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:32.461173 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dcace-c66e-4922-8c6c-8935c717db8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:32.462158 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dcace-c66e-4922-8c6c-8935c717db8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:32.467268 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65dcace-c66e-4922-8c6c-8935c717db8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:32.477299 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65dcace-c66e-4922-8c6c-8935c717db8e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:32.556356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 874/1732] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:32.560532 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e17c42-8af0-49f5-956e-dde5e7839715 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:32.587084 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e17c42-8af0-49f5-956e-dde5e7839715 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:32.605341 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf88f969-2f45-4fd6-aefa-b237ad15d8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:32.611574 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88f969-2f45-4fd6-aefa-b237ad15d8f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:32.611574 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88f969-2f45-4fd6-aefa-b237ad15d8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:32.611574 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88f969-2f45-4fd6-aefa-b237ad15d8f4 admin admin] PATH_INFO: `/v3/projects/4f6b7f56d4f24f13b3b935a9bf0e6efc/users/d8821c3b0f394f64909bdd9a46b407e0/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:32.708284 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf88f969-2f45-4fd6-aefa-b237ad15d8f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f6b7f56d4f24f13b3b935a9bf0e6efc, user_id=d8821c3b0f394f64909bdd9a46b407e0, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:32.721140 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf88f969-2f45-4fd6-aefa-b237ad15d8f4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:32.785526 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 859/1733] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:32 2022] PUT /identity/v3/projects/4f6b7f56d4f24f13b3b935a9bf0e6efc/users/d8821c3b0f394f64909bdd9a46b407e0/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:32.785816 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeafb9b-500c-40fb-ba16-0926e1e8cbec None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:32.786107 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeafb9b-500c-40fb-ba16-0926e1e8cbec None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:32.786379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeafb9b-500c-40fb-ba16-0926e1e8cbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:32.882344 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-3aeafb9b-500c-40fb-ba16-0926e1e8cbec None None] MFA Rules not processed for user `d8821c3b0f394f64909bdd9a46b407e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:32.883931 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3aeafb9b-500c-40fb-ba16-0926e1e8cbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:33.087659 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 875/1734] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:33.091553 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf88f969-2f45-4fd6-aefa-b237ad15d8f4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:33.259540 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:33.381072 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf88f969-2f45-4fd6-aefa-b237ad15d8f4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:33.397955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b2938-7eba-43b7-8d2b-2738919e6394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:33.398763 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b2938-7eba-43b7-8d2b-2738919e6394 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:33.399176 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b2938-7eba-43b7-8d2b-2738919e6394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:33.400115 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b2938-7eba-43b7-8d2b-2738919e6394 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:33.400693 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b2938-7eba-43b7-8d2b-2738919e6394 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:33.402913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b2938-7eba-43b7-8d2b-2738919e6394 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:33.503012 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 860/1735] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:33 2022] POST /identity/v3/projects => generated 375 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:33.504276 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeafb9b-500c-40fb-ba16-0926e1e8cbec None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:33.529314 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeafb9b-500c-40fb-ba16-0926e1e8cbec None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:33.757048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd591b-b9bd-4694-81a6-7312f5437ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:33.763575 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd591b-b9bd-4694-81a6-7312f5437ab9 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:33.765341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd591b-b9bd-4694-81a6-7312f5437ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:33.767577 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd591b-b9bd-4694-81a6-7312f5437ab9 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:33.768889 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd591b-b9bd-4694-81a6-7312f5437ab9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:33.776821 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd591b-b9bd-4694-81a6-7312f5437ab9 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:33.794451 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:33.836974 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 876/1736] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:33 2022] POST /identity/v3/users => generated 294 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:33.847937 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b2938-7eba-43b7-8d2b-2738919e6394 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:33.926793 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b2938-7eba-43b7-8d2b-2738919e6394 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:33.931743 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc70496-78bf-4835-8992-32cdc6c3bef2 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:33.932038 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc70496-78bf-4835-8992-32cdc6c3bef2 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:33.932321 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc70496-78bf-4835-8992-32cdc6c3bef2 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:33.934214 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 877/1737] 10.222.0.81 () {54 vars in 921 bytes} [Tue Jun 7 14:21:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:21:33.967198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c932f96-7f19-4543-aeac-c20da1bcf13f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:33.967584 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c932f96-7f19-4543-aeac-c20da1bcf13f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:33.967930 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c932f96-7f19-4543-aeac-c20da1bcf13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:33.978478 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507f404-e0b9-431b-b3e3-a0ac782b077c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:33.991459 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507f404-e0b9-431b-b3e3-a0ac782b077c admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:33.991459 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507f404-e0b9-431b-b3e3-a0ac782b077c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:33.991459 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507f404-e0b9-431b-b3e3-a0ac782b077c admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:33.995334 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507f404-e0b9-431b-b3e3-a0ac782b077c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:34.009161 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507f404-e0b9-431b-b3e3-a0ac782b077c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:34.047259 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-5c932f96-7f19-4543-aeac-c20da1bcf13f None None] MFA Rules not processed for user `ab73cf3febbf4c768cd0116306f16c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:34.049737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5c932f96-7f19-4543-aeac-c20da1bcf13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:34.102685 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 861/1738] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:34.103883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507f404-e0b9-431b-b3e3-a0ac782b077c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:34.105912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2507f404-e0b9-431b-b3e3-a0ac782b077c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:34.137153 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507f404-e0b9-431b-b3e3-a0ac782b077c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:34.145706 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba3e58-a502-4a29-8e99-7b1d9ad8fe95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:34.147161 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba3e58-a502-4a29-8e99-7b1d9ad8fe95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:34.147609 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba3e58-a502-4a29-8e99-7b1d9ad8fe95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:34.147861 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba3e58-a502-4a29-8e99-7b1d9ad8fe95 admin admin] PATH_INFO: `/v3/users/0cb00f50054442749dc681eb800e8117` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:34.149819 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba3e58-a502-4a29-8e99-7b1d9ad8fe95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cb00f50054442749dc681eb800e8117)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:34.183244 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba3e58-a502-4a29-8e99-7b1d9ad8fe95 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:34.281298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-31ba3e58-a502-4a29-8e99-7b1d9ad8fe95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cb00f50054442749dc681eb800e8117'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:34.282221 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-31ba3e58-a502-4a29-8e99-7b1d9ad8fe95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cb00f50054442749dc681eb800e8117'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:34.284788 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-31ba3e58-a502-4a29-8e99-7b1d9ad8fe95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cb00f50054442749dc681eb800e8117'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:34.308238 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-31ba3e58-a502-4a29-8e99-7b1d9ad8fe95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cb00f50054442749dc681eb800e8117'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:34.338935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 878/1739] 10.222.0.81 () {58 vars in 1018 bytes} [Tue Jun 7 14:21:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:34.342654 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c932f96-7f19-4543-aeac-c20da1bcf13f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:34.345228 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5c932f96-7f19-4543-aeac-c20da1bcf13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:34.367390 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-31ba3e58-a502-4a29-8e99-7b1d9ad8fe95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cb00f50054442749dc681eb800e8117'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:34.416924 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 862/1740] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:21:34 2022] DELETE /identity/v3/users/0cb00f50054442749dc681eb800e8117 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:34.417927 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c932f96-7f19-4543-aeac-c20da1bcf13f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:34.418340 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba3e58-a502-4a29-8e99-7b1d9ad8fe95 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:34.420452 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-31ba3e58-a502-4a29-8e99-7b1d9ad8fe95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:34.622911 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1087ade-c21b-4ee9-8a3e-bc9654c836b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:34.623534 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1087ade-c21b-4ee9-8a3e-bc9654c836b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:34.624497 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1087ade-c21b-4ee9-8a3e-bc9654c836b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:34.624683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1087ade-c21b-4ee9-8a3e-bc9654c836b9 admin admin] PATH_INFO: `/v3/projects/85df0f73285b4d2fb84b0d6305996fbd/users/108f9071ce904e8fb10997ceb84c5857/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:34.710649 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1087ade-c21b-4ee9-8a3e-bc9654c836b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85df0f73285b4d2fb84b0d6305996fbd, user_id=108f9071ce904e8fb10997ceb84c5857, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:34.732675 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba3e58-a502-4a29-8e99-7b1d9ad8fe95 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:34.734632 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1087ade-c21b-4ee9-8a3e-bc9654c836b9 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:34.767598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97cdbc3-8785-48ef-bb6a-019476efadf2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab73cf3febbf4c768cd0116306f16c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:34.768289 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97cdbc3-8785-48ef-bb6a-019476efadf2 service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:34.768783 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97cdbc3-8785-48ef-bb6a-019476efadf2 service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:34.769033 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97cdbc3-8785-48ef-bb6a-019476efadf2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:34.793546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 879/1741] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:34 2022] PUT /identity/v3/projects/85df0f73285b4d2fb84b0d6305996fbd/users/108f9071ce904e8fb10997ceb84c5857/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:34.797331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679ddfb-0050-4453-9f29-7f9537e63394 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:34.805738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679ddfb-0050-4453-9f29-7f9537e63394 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:34.806635 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679ddfb-0050-4453-9f29-7f9537e63394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:34.826166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97cdbc3-8785-48ef-bb6a-019476efadf2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:34.828973 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97cdbc3-8785-48ef-bb6a-019476efadf2 service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:34.868694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-5679ddfb-0050-4453-9f29-7f9537e63394 None None] MFA Rules not processed for user `d20025ab12f4450da20c8e41b931b5fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:34.871558 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5679ddfb-0050-4453-9f29-7f9537e63394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:34.874928 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 863/1742] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:34.875692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97cdbc3-8785-48ef-bb6a-019476efadf2 service placement] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:34.909171 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97cdbc3-8785-48ef-bb6a-019476efadf2 service placement] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:34.920629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:35.113184 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 880/1743] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:35.130642 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc03e9-b661-4360-85de-3fc22cbf31d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:35.131567 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc03e9-b661-4360-85de-3fc22cbf31d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:35.131855 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc03e9-b661-4360-85de-3fc22cbf31d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:35.132136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc03e9-b661-4360-85de-3fc22cbf31d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:35.136373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc03e9-b661-4360-85de-3fc22cbf31d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:35.138215 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc03e9-b661-4360-85de-3fc22cbf31d2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:35.204209 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5679ddfb-0050-4453-9f29-7f9537e63394 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:35.206333 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5679ddfb-0050-4453-9f29-7f9537e63394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:35.221067 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 864/1744] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:35.227831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc03e9-b661-4360-85de-3fc22cbf31d2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:35.248173 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5679ddfb-0050-4453-9f29-7f9537e63394 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:35.267995 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc03e9-b661-4360-85de-3fc22cbf31d2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:35.292159 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee4cff-c319-4eda-9f07-11dea96f30ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:35.293501 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee4cff-c319-4eda-9f07-11dea96f30ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:35.296611 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee4cff-c319-4eda-9f07-11dea96f30ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:35.297037 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee4cff-c319-4eda-9f07-11dea96f30ef admin admin] PATH_INFO: `/v3/projects/85df0f73285b4d2fb84b0d6305996fbd/users/108f9071ce904e8fb10997ceb84c5857/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:35.309392 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aee4cff-c319-4eda-9f07-11dea96f30ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=85df0f73285b4d2fb84b0d6305996fbd, user_id=108f9071ce904e8fb10997ceb84c5857, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:35.311796 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85039e-f315-490d-b453-ffe5844016b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:35.312951 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85039e-f315-490d-b453-ffe5844016b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:35.313293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85039e-f315-490d-b453-ffe5844016b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:35.313552 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85039e-f315-490d-b453-ffe5844016b3 admin admin] PATH_INFO: `/v3/projects/bcc308a8cf1f4e75ba30dfd1c84dbbfe` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:35.317724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c85039e-f315-490d-b453-ffe5844016b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcc308a8cf1f4e75ba30dfd1c84dbbfe)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:35.323500 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aee4cff-c319-4eda-9f07-11dea96f30ef admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:35.331673 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c85039e-f315-490d-b453-ffe5844016b3 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:35.375107 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 865/1745] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:35 2022] PUT /identity/v3/projects/85df0f73285b4d2fb84b0d6305996fbd/users/108f9071ce904e8fb10997ceb84c5857/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:35.392638 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09960737-48b5-4bd1-94b5-ce8ad9faca5d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:35.393227 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09960737-48b5-4bd1-94b5-ce8ad9faca5d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:35.394264 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:35.394832 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09960737-48b5-4bd1-94b5-ce8ad9faca5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:35.510590 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-09960737-48b5-4bd1-94b5-ce8ad9faca5d None None] MFA Rules not processed for user `108f9071ce904e8fb10997ceb84c5857`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:35.512228 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-09960737-48b5-4bd1-94b5-ce8ad9faca5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:35.631714 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4c85039e-f315-490d-b453-ffe5844016b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcc308a8cf1f4e75ba30dfd1c84dbbfe', 'request_id': 'req-4c85039e-f315-490d-b453-ffe5844016b3'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:35.660872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4c85039e-f315-490d-b453-ffe5844016b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcc308a8cf1f4e75ba30dfd1c84dbbfe', 'request_id': 'req-4c85039e-f315-490d-b453-ffe5844016b3'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:35.683128 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4c85039e-f315-490d-b453-ffe5844016b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcc308a8cf1f4e75ba30dfd1c84dbbfe', 'request_id': 'req-4c85039e-f315-490d-b453-ffe5844016b3'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:35.725786 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4c85039e-f315-490d-b453-ffe5844016b3 admin admin] The token cache is being invalidate because project bcc308a8cf1f4e75ba30dfd1c84dbbfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:21:35.726289 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4c85039e-f315-490d-b453-ffe5844016b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:35.728909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4c85039e-f315-490d-b453-ffe5844016b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:35.734579 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 881/1746] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:21:35 2022] DELETE /identity/v3/projects/bcc308a8cf1f4e75ba30dfd1c84dbbfe => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:35.738565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85039e-f315-490d-b453-ffe5844016b3 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:35.740150 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4c85039e-f315-490d-b453-ffe5844016b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:35.824750 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 866/1747] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:35 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 430 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:35.834879 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09960737-48b5-4bd1-94b5-ce8ad9faca5d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:35.836892 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-09960737-48b5-4bd1-94b5-ce8ad9faca5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:35.946152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85039e-f315-490d-b453-ffe5844016b3 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:35.953955 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14311e21-f6fb-4eb5-a1c3-6ec88242c1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:35.955607 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14311e21-f6fb-4eb5-a1c3-6ec88242c1db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:35.956042 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14311e21-f6fb-4eb5-a1c3-6ec88242c1db admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:35.956450 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14311e21-f6fb-4eb5-a1c3-6ec88242c1db admin admin] PATH_INFO: `/v3/users/d7192f815ea44e7f806c1b8d487711d9` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:35.958186 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14311e21-f6fb-4eb5-a1c3-6ec88242c1db admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7192f815ea44e7f806c1b8d487711d9)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:35.961046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14311e21-f6fb-4eb5-a1c3-6ec88242c1db admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:36.097397 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09960737-48b5-4bd1-94b5-ce8ad9faca5d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:36.121558 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-14311e21-f6fb-4eb5-a1c3-6ec88242c1db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7192f815ea44e7f806c1b8d487711d9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:36.167208 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-14311e21-f6fb-4eb5-a1c3-6ec88242c1db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7192f815ea44e7f806c1b8d487711d9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:36.169780 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-14311e21-f6fb-4eb5-a1c3-6ec88242c1db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7192f815ea44e7f806c1b8d487711d9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:36.171521 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-14311e21-f6fb-4eb5-a1c3-6ec88242c1db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7192f815ea44e7f806c1b8d487711d9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:36.178800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:36.215340 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-14311e21-f6fb-4eb5-a1c3-6ec88242c1db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7192f815ea44e7f806c1b8d487711d9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:36.235412 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f465c1-d206-4031-9088-5527e1bc8e43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:36.238726 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f465c1-d206-4031-9088-5527e1bc8e43 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:36.240300 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f465c1-d206-4031-9088-5527e1bc8e43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:36.240684 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f465c1-d206-4031-9088-5527e1bc8e43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:36.244817 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d8f465c1-d206-4031-9088-5527e1bc8e43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:36.257278 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 882/1748] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:21:35 2022] DELETE /identity/v3/users/d7192f815ea44e7f806c1b8d487711d9 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:36.446255 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f465c1-d206-4031-9088-5527e1bc8e43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:36.454773 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f465c1-d206-4031-9088-5527e1bc8e43 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:36.485197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 867/1749] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 651 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:36.663087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69bd2e-e616-4f59-8be2-ce2d205231ff None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:36.663629 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69bd2e-e616-4f59-8be2-ce2d205231ff None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:36.663704 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69bd2e-e616-4f59-8be2-ce2d205231ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:36.723338 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-5c69bd2e-e616-4f59-8be2-ce2d205231ff None None] MFA Rules not processed for user `a6a08eaecbcd41228571739d8387882a`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:36.729306 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5c69bd2e-e616-4f59-8be2-ce2d205231ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:36.784630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:36.951679 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 883/1750] 10.222.0.75 () {58 vars in 1015 bytes} [Tue Jun 7 14:21:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:37.044884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f465c1-d206-4031-9088-5527e1bc8e43 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:37.047443 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d8f465c1-d206-4031-9088-5527e1bc8e43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:37.143361 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c69bd2e-e616-4f59-8be2-ce2d205231ff None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:37.145608 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5c69bd2e-e616-4f59-8be2-ce2d205231ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:37.207250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f465c1-d206-4031-9088-5527e1bc8e43 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:37.222328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1300c94-eef6-47d8-b90d-b4e2a032635f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:37.228620 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1300c94-eef6-47d8-b90d-b4e2a032635f service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:37.228875 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1300c94-eef6-47d8-b90d-b4e2a032635f service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:37.231376 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1300c94-eef6-47d8-b90d-b4e2a032635f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:37.266356 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1300c94-eef6-47d8-b90d-b4e2a032635f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:37.270272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1300c94-eef6-47d8-b90d-b4e2a032635f service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:37.301062 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 868/1751] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:37.365694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c69bd2e-e616-4f59-8be2-ce2d205231ff None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:37.385700 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c7513-16ff-485e-be1a-9d692b1c8bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:37.387847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c7513-16ff-485e-be1a-9d692b1c8bc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:37.389292 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c7513-16ff-485e-be1a-9d692b1c8bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:37.389938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c7513-16ff-485e-be1a-9d692b1c8bc5 admin admin] PATH_INFO: `/v3/projects/152caa2f11af4b8ebb87b6616ae9c49b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:37.394979 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c7513-16ff-485e-be1a-9d692b1c8bc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=152caa2f11af4b8ebb87b6616ae9c49b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:37.401079 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c7513-16ff-485e-be1a-9d692b1c8bc5 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:37.547485 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f32c7513-16ff-485e-be1a-9d692b1c8bc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '152caa2f11af4b8ebb87b6616ae9c49b', 'request_id': 'req-f32c7513-16ff-485e-be1a-9d692b1c8bc5'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:37.570614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f32c7513-16ff-485e-be1a-9d692b1c8bc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '152caa2f11af4b8ebb87b6616ae9c49b', 'request_id': 'req-f32c7513-16ff-485e-be1a-9d692b1c8bc5'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:37.587387 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f32c7513-16ff-485e-be1a-9d692b1c8bc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '152caa2f11af4b8ebb87b6616ae9c49b', 'request_id': 'req-f32c7513-16ff-485e-be1a-9d692b1c8bc5'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:37.617209 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f32c7513-16ff-485e-be1a-9d692b1c8bc5 admin admin] The token cache is being invalidate because project 152caa2f11af4b8ebb87b6616ae9c49b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:21:37.617507 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f32c7513-16ff-485e-be1a-9d692b1c8bc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:37.620193 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f32c7513-16ff-485e-be1a-9d692b1c8bc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:37.628981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 884/1752] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:21:37 2022] DELETE /identity/v3/projects/152caa2f11af4b8ebb87b6616ae9c49b => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:21:37.630936 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1300c94-eef6-47d8-b90d-b4e2a032635f service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:37.646837 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1300c94-eef6-47d8-b90d-b4e2a032635f service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:37.859311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ec6f73-c50c-4f2c-a9f6-341b2e0a408c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:37.860264 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec6f73-c50c-4f2c-a9f6-341b2e0a408c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:37.860551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec6f73-c50c-4f2c-a9f6-341b2e0a408c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:37.860820 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec6f73-c50c-4f2c-a9f6-341b2e0a408c admin admin] PATH_INFO: `/v3/users/881c17d6099b4c70a705a096b273025f` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:37.862612 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ec6f73-c50c-4f2c-a9f6-341b2e0a408c admin admin] RBAC: Authorizing `identity:delete_user(user_id=881c17d6099b4c70a705a096b273025f)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:37.868339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ec6f73-c50c-4f2c-a9f6-341b2e0a408c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:38.014957 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:38.093215 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-11ec6f73-c50c-4f2c-a9f6-341b2e0a408c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '881c17d6099b4c70a705a096b273025f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:38.094960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-11ec6f73-c50c-4f2c-a9f6-341b2e0a408c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '881c17d6099b4c70a705a096b273025f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:38.098784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-11ec6f73-c50c-4f2c-a9f6-341b2e0a408c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '881c17d6099b4c70a705a096b273025f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:38.114373 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c7513-16ff-485e-be1a-9d692b1c8bc5 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:38.116483 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f32c7513-16ff-485e-be1a-9d692b1c8bc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:38.168972 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-11ec6f73-c50c-4f2c-a9f6-341b2e0a408c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '881c17d6099b4c70a705a096b273025f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:38.185182 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c7513-16ff-485e-be1a-9d692b1c8bc5 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:38.207050 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ccf873-755f-4cec-a9f3-8bc0db4e08c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:38.211807 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccf873-755f-4cec-a9f3-8bc0db4e08c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:38.213053 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccf873-755f-4cec-a9f3-8bc0db4e08c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:38.215101 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccf873-755f-4cec-a9f3-8bc0db4e08c8 admin admin] PATH_INFO: `/v3/projects/99ac6fdcef584017ad990af7f33c7033` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:38.220194 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ccf873-755f-4cec-a9f3-8bc0db4e08c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99ac6fdcef584017ad990af7f33c7033)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:38.227566 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ccf873-755f-4cec-a9f3-8bc0db4e08c8 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:38.247816 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-11ec6f73-c50c-4f2c-a9f6-341b2e0a408c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '881c17d6099b4c70a705a096b273025f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:38.323596 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c6ccf873-755f-4cec-a9f3-8bc0db4e08c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99ac6fdcef584017ad990af7f33c7033', 'request_id': 'req-c6ccf873-755f-4cec-a9f3-8bc0db4e08c8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:38.324583 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 869/1753] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:21:37 2022] DELETE /identity/v3/users/881c17d6099b4c70a705a096b273025f => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:38.326634 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ec6f73-c50c-4f2c-a9f6-341b2e0a408c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:38.327797 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:38.333531 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-11ec6f73-c50c-4f2c-a9f6-341b2e0a408c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:38.354889 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c6ccf873-755f-4cec-a9f3-8bc0db4e08c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99ac6fdcef584017ad990af7f33c7033', 'request_id': 'req-c6ccf873-755f-4cec-a9f3-8bc0db4e08c8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:38.366301 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c6ccf873-755f-4cec-a9f3-8bc0db4e08c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99ac6fdcef584017ad990af7f33c7033', 'request_id': 'req-c6ccf873-755f-4cec-a9f3-8bc0db4e08c8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:38.415246 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c6ccf873-755f-4cec-a9f3-8bc0db4e08c8 admin admin] The token cache is being invalidate because project 99ac6fdcef584017ad990af7f33c7033 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:21:38.415471 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c6ccf873-755f-4cec-a9f3-8bc0db4e08c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:38.418282 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c6ccf873-755f-4cec-a9f3-8bc0db4e08c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:38.421230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 885/1754] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:21:38 2022] DELETE /identity/v3/projects/99ac6fdcef584017ad990af7f33c7033 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:38.426847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b23bf-b9f0-46ec-acc4-f65a9cafbfda None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:38.427142 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b23bf-b9f0-46ec-acc4-f65a9cafbfda None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:38.427458 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b23bf-b9f0-46ec-acc4-f65a9cafbfda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:38.505882 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-662b23bf-b9f0-46ec-acc4-f65a9cafbfda None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:38.515430 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-662b23bf-b9f0-46ec-acc4-f65a9cafbfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:38.648489 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ec6f73-c50c-4f2c-a9f6-341b2e0a408c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:38.764208 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 886/1755] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:38.766084 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7c87c-b950-4a27-96c5-0d4c815f84a7 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:38.766386 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7c87c-b950-4a27-96c5-0d4c815f84a7 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:38.766683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7c87c-b950-4a27-96c5-0d4c815f84a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:38.817984 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a0a7c87c-b950-4a27-96c5-0d4c815f84a7 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:38.824894 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a0a7c87c-b950-4a27-96c5-0d4c815f84a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:38.861226 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 887/1756] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:21:38.862411 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7c87c-b950-4a27-96c5-0d4c815f84a7 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:38.892708 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7c87c-b950-4a27-96c5-0d4c815f84a7 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:38.900650 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7bd74-ae61-4040-9a59-59d70fbf3a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:38.901132 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7bd74-ae61-4040-9a59-59d70fbf3a45 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:38.902053 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7bd74-ae61-4040-9a59-59d70fbf3a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:38.902490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7bd74-ae61-4040-9a59-59d70fbf3a45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:38.903956 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7bd74-ae61-4040-9a59-59d70fbf3a45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:38.904277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7bd74-ae61-4040-9a59-59d70fbf3a45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:38.907085 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb5a61f-2453-4f6a-8a85-4e766240174a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:38.907328 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7bd74-ae61-4040-9a59-59d70fbf3a45 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:38.917697 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5a61f-2453-4f6a-8a85-4e766240174a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:38.917906 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5a61f-2453-4f6a-8a85-4e766240174a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:38.918103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5a61f-2453-4f6a-8a85-4e766240174a admin admin] PATH_INFO: `/v3/projects/59eff67373ac4a7091ee20065a20bea2` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:38.920315 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb5a61f-2453-4f6a-8a85-4e766240174a admin admin] RBAC: Authorizing `identity:delete_project(project_id=59eff67373ac4a7091ee20065a20bea2)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:38.927572 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb5a61f-2453-4f6a-8a85-4e766240174a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:38.956290 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 888/1757] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:38.956290 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7bd74-ae61-4040-9a59-59d70fbf3a45 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:38.994623 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7bd74-ae61-4040-9a59-59d70fbf3a45 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:39.018247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6df36f-a40f-4d81-9941-aca447f9d44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:39.020781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6df36f-a40f-4d81-9941-aca447f9d44e admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:39.022300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6df36f-a40f-4d81-9941-aca447f9d44e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:39.028928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6df36f-a40f-4d81-9941-aca447f9d44e admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:39.032974 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6df36f-a40f-4d81-9941-aca447f9d44e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:39.039114 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6df36f-a40f-4d81-9941-aca447f9d44e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:39.148507 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 889/1758] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:38 2022] POST /identity/v3/projects => generated 379 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:39.149334 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6df36f-a40f-4d81-9941-aca447f9d44e admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:39.167269 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9cb5a61f-2453-4f6a-8a85-4e766240174a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59eff67373ac4a7091ee20065a20bea2', 'request_id': 'req-9cb5a61f-2453-4f6a-8a85-4e766240174a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:39.172267 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0f6df36f-a40f-4d81-9941-aca447f9d44e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:39.196914 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9cb5a61f-2453-4f6a-8a85-4e766240174a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59eff67373ac4a7091ee20065a20bea2', 'request_id': 'req-9cb5a61f-2453-4f6a-8a85-4e766240174a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:39.200310 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:39.210072 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9cb5a61f-2453-4f6a-8a85-4e766240174a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59eff67373ac4a7091ee20065a20bea2', 'request_id': 'req-9cb5a61f-2453-4f6a-8a85-4e766240174a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:39.248294 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9cb5a61f-2453-4f6a-8a85-4e766240174a admin admin] The token cache is being invalidate because project 59eff67373ac4a7091ee20065a20bea2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:21:39.249991 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9cb5a61f-2453-4f6a-8a85-4e766240174a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:39.252820 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9cb5a61f-2453-4f6a-8a85-4e766240174a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:39.259567 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 870/1759] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:21:38 2022] DELETE /identity/v3/projects/59eff67373ac4a7091ee20065a20bea2 => generated 0 bytes in 934 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:39.261404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb5a61f-2453-4f6a-8a85-4e766240174a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:39.267259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9cb5a61f-2453-4f6a-8a85-4e766240174a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:39.493137 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6df36f-a40f-4d81-9941-aca447f9d44e admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:39.504217 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f3e2f-ede7-488a-832d-7a21835d2944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:39.505292 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb5a61f-2453-4f6a-8a85-4e766240174a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:39.505676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f3e2f-ede7-488a-832d-7a21835d2944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:39.506234 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f3e2f-ede7-488a-832d-7a21835d2944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:39.506735 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f3e2f-ede7-488a-832d-7a21835d2944 admin admin] PATH_INFO: `/v3/projects/49f7cc86181f4d8aa35593d8f6339e42` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:39.508582 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f3e2f-ede7-488a-832d-7a21835d2944 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49f7cc86181f4d8aa35593d8f6339e42)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:39.511697 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f3e2f-ede7-488a-832d-7a21835d2944 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:39.515924 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8780c-23f8-4231-b3f0-709fa9b0b323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:39.516667 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8780c-23f8-4231-b3f0-709fa9b0b323 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:39.516934 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8780c-23f8-4231-b3f0-709fa9b0b323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:39.517199 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8780c-23f8-4231-b3f0-709fa9b0b323 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:39.517904 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8780c-23f8-4231-b3f0-709fa9b0b323 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:39.528193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8780c-23f8-4231-b3f0-709fa9b0b323 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:39.608733 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 871/1760] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:39 2022] POST /identity/v3/users => generated 290 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:39.610100 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b31676-0003-4af4-8496-f1de9af8bbe3 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:39.610391 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b31676-0003-4af4-8496-f1de9af8bbe3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:39.610691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b31676-0003-4af4-8496-f1de9af8bbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:39.646229 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:INSERT=2,keystone:DELETE=6,keystone:SELECT=92 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:39.664779 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-78b31676-0003-4af4-8496-f1de9af8bbe3 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:39.666038 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-78b31676-0003-4af4-8496-f1de9af8bbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:39.666289 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f32f3e2f-ede7-488a-832d-7a21835d2944 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49f7cc86181f4d8aa35593d8f6339e42', 'request_id': 'req-f32f3e2f-ede7-488a-832d-7a21835d2944'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:39.699386 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f32f3e2f-ede7-488a-832d-7a21835d2944 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49f7cc86181f4d8aa35593d8f6339e42', 'request_id': 'req-f32f3e2f-ede7-488a-832d-7a21835d2944'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:39.731271 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f32f3e2f-ede7-488a-832d-7a21835d2944 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49f7cc86181f4d8aa35593d8f6339e42', 'request_id': 'req-f32f3e2f-ede7-488a-832d-7a21835d2944'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:39.781037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f32f3e2f-ede7-488a-832d-7a21835d2944 admin admin] The token cache is being invalidate because project 49f7cc86181f4d8aa35593d8f6339e42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:21:39.781798 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f32f3e2f-ede7-488a-832d-7a21835d2944 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:39.784604 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f32f3e2f-ede7-488a-832d-7a21835d2944 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:39.792798 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 890/1761] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:21:39 2022] DELETE /identity/v3/projects/49f7cc86181f4d8aa35593d8f6339e42 => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:39.795105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a58a72-7947-4f72-810c-d188f0e195ea None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:39.796673 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a58a72-7947-4f72-810c-d188f0e195ea None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:39.797102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a58a72-7947-4f72-810c-d188f0e195ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:39.825721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 872/1762] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:39.826979 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b31676-0003-4af4-8496-f1de9af8bbe3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:39.829165 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-78b31676-0003-4af4-8496-f1de9af8bbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:39.866205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b31676-0003-4af4-8496-f1de9af8bbe3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:39.871829 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-07a58a72-7947-4f72-810c-d188f0e195ea None None] MFA Rules not processed for user `6592aab6435242dca77b854a74ced4f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:39.873366 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-07a58a72-7947-4f72-810c-d188f0e195ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:39.878970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3ee1c-c136-4e89-b1aa-ea7712cf9120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:39.881659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3ee1c-c136-4e89-b1aa-ea7712cf9120 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:39.882171 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3ee1c-c136-4e89-b1aa-ea7712cf9120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:39.882724 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3ee1c-c136-4e89-b1aa-ea7712cf9120 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:39.884149 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c3ee1c-c136-4e89-b1aa-ea7712cf9120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:39.888451 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c3ee1c-c136-4e89-b1aa-ea7712cf9120 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:39.902616 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 873/1763] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:39.903855 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3ee1c-c136-4e89-b1aa-ea7712cf9120 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:39.906396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a5c3ee1c-c136-4e89-b1aa-ea7712cf9120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:39.921805 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3ee1c-c136-4e89-b1aa-ea7712cf9120 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:39.927183 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2104f0a-fcd6-4a1f-ab82-1f1e17272cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:39.928652 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2104f0a-fcd6-4a1f-ab82-1f1e17272cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:39.929163 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2104f0a-fcd6-4a1f-ab82-1f1e17272cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:39.929650 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2104f0a-fcd6-4a1f-ab82-1f1e17272cb9 admin admin] PATH_INFO: `/v3/users/dc7a2cbd571e45d2bc7791cc62f7c124` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:39.931140 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2104f0a-fcd6-4a1f-ab82-1f1e17272cb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc7a2cbd571e45d2bc7791cc62f7c124)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:39.933727 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2104f0a-fcd6-4a1f-ab82-1f1e17272cb9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:40.020368 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a2104f0a-fcd6-4a1f-ab82-1f1e17272cb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc7a2cbd571e45d2bc7791cc62f7c124'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:40.021244 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a2104f0a-fcd6-4a1f-ab82-1f1e17272cb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc7a2cbd571e45d2bc7791cc62f7c124'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:40.022129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a2104f0a-fcd6-4a1f-ab82-1f1e17272cb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc7a2cbd571e45d2bc7791cc62f7c124'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:40.035871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a2104f0a-fcd6-4a1f-ab82-1f1e17272cb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc7a2cbd571e45d2bc7791cc62f7c124'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:40.041207 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:40.042073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 891/1764] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:40.043427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49b8a4-af17-460a-9761-c46dc05f6bd4 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:40.043750 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49b8a4-af17-460a-9761-c46dc05f6bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:40.044131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49b8a4-af17-460a-9761-c46dc05f6bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:40.057120 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a2104f0a-fcd6-4a1f-ab82-1f1e17272cb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc7a2cbd571e45d2bc7791cc62f7c124'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:40.070100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-cc49b8a4-af17-460a-9761-c46dc05f6bd4 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:40.071348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-cc49b8a4-af17-460a-9761-c46dc05f6bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:40.074262 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 874/1765] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:21:39 2022] DELETE /identity/v3/users/dc7a2cbd571e45d2bc7791cc62f7c124 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:40.078708 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63bf92-246e-4321-bded-11e402e8c1e8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:40.079001 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63bf92-246e-4321-bded-11e402e8c1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:40.079311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63bf92-246e-4321-bded-11e402e8c1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:40.102252 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3c63bf92-246e-4321-bded-11e402e8c1e8 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:40.103931 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3c63bf92-246e-4321-bded-11e402e8c1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:40.147561 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 892/1766] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:40.148689 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49b8a4-af17-460a-9761-c46dc05f6bd4 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:40.149633 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-cc49b8a4-af17-460a-9761-c46dc05f6bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:40.176263 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49b8a4-af17-460a-9761-c46dc05f6bd4 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:40.178306 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 875/1767] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:40.179799 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da2a82-6bac-4160-8573-cce578258df3 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:40.180026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da2a82-6bac-4160-8573-cce578258df3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:40.180312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da2a82-6bac-4160-8573-cce578258df3 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:40.181035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ee69e-0474-4809-a3a4-f8004620cc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:40.181707 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 876/1768] 10.222.0.81 () {54 vars in 921 bytes} [Tue Jun 7 14:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:40.182005 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ee69e-0474-4809-a3a4-f8004620cc67 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:40.182296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ee69e-0474-4809-a3a4-f8004620cc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:40.182478 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da2a82-6bac-4160-8573-cce578258df3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:40.182575 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ee69e-0474-4809-a3a4-f8004620cc67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:40.183542 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0ee69e-0474-4809-a3a4-f8004620cc67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:40.184189 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-40da2a82-6bac-4160-8573-cce578258df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:40.185176 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0ee69e-0474-4809-a3a4-f8004620cc67 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:40.200621 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 893/1769] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:40.202896 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ba738-1589-4b70-8799-b7184cff5c68 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:40.203330 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ba738-1589-4b70-8799-b7184cff5c68 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:40.203726 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ba738-1589-4b70-8799-b7184cff5c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:40.226427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-2b4ba738-1589-4b70-8799-b7184cff5c68 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:40.227843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2b4ba738-1589-4b70-8799-b7184cff5c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:40.246407 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 894/1770] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:40.247940 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ed95c-2077-45fc-b1aa-9b49e9254c06 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:40.248329 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ed95c-2077-45fc-b1aa-9b49e9254c06 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:40.249270 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ed95c-2077-45fc-b1aa-9b49e9254c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:40.279926 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-f73ed95c-2077-45fc-b1aa-9b49e9254c06 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:40.281404 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f73ed95c-2077-45fc-b1aa-9b49e9254c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:40.291812 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da2a82-6bac-4160-8573-cce578258df3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:40.294302 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 895/1771] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:40.295211 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73ed95c-2077-45fc-b1aa-9b49e9254c06 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:40.299314 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b75341d-ec75-4a53-af37-2d0ada267c4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:40.300075 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75341d-ec75-4a53-af37-2d0ada267c4b service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:40.300424 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75341d-ec75-4a53-af37-2d0ada267c4b service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:40.300882 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75341d-ec75-4a53-af37-2d0ada267c4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:40.305835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73ed95c-2077-45fc-b1aa-9b49e9254c06 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:40.311949 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0232a5-8615-45ed-9122-020da11aa39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:40.312804 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0232a5-8615-45ed-9122-020da11aa39e admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:40.314217 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0232a5-8615-45ed-9122-020da11aa39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:40.314217 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0232a5-8615-45ed-9122-020da11aa39e admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:40.314400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0232a5-8615-45ed-9122-020da11aa39e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:40.314638 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0232a5-8615-45ed-9122-020da11aa39e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:40.316444 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0232a5-8615-45ed-9122-020da11aa39e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:40.333297 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 896/1772] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:40.333297 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40e0ba-9f7e-4e94-8da8-c315727e87fa None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:40.333297 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40e0ba-9f7e-4e94-8da8-c315727e87fa None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:40.333297 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40e0ba-9f7e-4e94-8da8-c315727e87fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:40.355583 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-8c40e0ba-9f7e-4e94-8da8-c315727e87fa None None] MFA Rules not processed for user `2ae726fbc9cf40e68c7587552a118a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:40.356850 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8c40e0ba-9f7e-4e94-8da8-c315727e87fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:40.367205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 897/1773] 10.222.0.81 () {58 vars in 1018 bytes} [Tue Jun 7 14:21:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:40.368155 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c40e0ba-9f7e-4e94-8da8-c315727e87fa None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:40.379973 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b75341d-ec75-4a53-af37-2d0ada267c4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:40.380895 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c40e0ba-9f7e-4e94-8da8-c315727e87fa None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:40.382264 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b75341d-ec75-4a53-af37-2d0ada267c4b service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:40.386745 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1836c7b-ed31-413f-867c-e60683fd24dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:40.389618 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1836c7b-ed31-413f-867c-e60683fd24dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:40.390113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1836c7b-ed31-413f-867c-e60683fd24dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:40.390462 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1836c7b-ed31-413f-867c-e60683fd24dd admin admin] PATH_INFO: `/v3/projects/cec5883c84d642c9a8fe2818636b7a90/users/b6055476a66646e9b493e70a62931e71/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:40.394738 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 877/1774] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:40.395708 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b75341d-ec75-4a53-af37-2d0ada267c4b service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:40.397205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1b75341d-ec75-4a53-af37-2d0ada267c4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:40.404585 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:40.412137 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1836c7b-ed31-413f-867c-e60683fd24dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cec5883c84d642c9a8fe2818636b7a90, user_id=b6055476a66646e9b493e70a62931e71, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:40.417305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1836c7b-ed31-413f-867c-e60683fd24dd admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:40.426234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:40.445826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 898/1775] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:40 2022] PUT /identity/v3/projects/cec5883c84d642c9a8fe2818636b7a90/users/b6055476a66646e9b493e70a62931e71/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:40.446971 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1836c7b-ed31-413f-867c-e60683fd24dd admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:40.487613 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b75341d-ec75-4a53-af37-2d0ada267c4b service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:40.540818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1836c7b-ed31-413f-867c-e60683fd24dd admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:40.548431 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28542384-1fdc-4eb6-b8f0-9494d4f56335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:40.549318 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5e759-b3d7-4e22-b7eb-7403fb87da93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:40.549991 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5e759-b3d7-4e22-b7eb-7403fb87da93 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:40.550282 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5e759-b3d7-4e22-b7eb-7403fb87da93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:40.550560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5e759-b3d7-4e22-b7eb-7403fb87da93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:40.551493 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28542384-1fdc-4eb6-b8f0-9494d4f56335 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:40.552085 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28542384-1fdc-4eb6-b8f0-9494d4f56335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:40.552949 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28542384-1fdc-4eb6-b8f0-9494d4f56335 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:40.554031 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28542384-1fdc-4eb6-b8f0-9494d4f56335 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:40.554552 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28542384-1fdc-4eb6-b8f0-9494d4f56335 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:40.560700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee5e759-b3d7-4e22-b7eb-7403fb87da93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:40.563320 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee5e759-b3d7-4e22-b7eb-7403fb87da93 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:40.564285 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28542384-1fdc-4eb6-b8f0-9494d4f56335 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:40.575115 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 878/1776] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:40.575936 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5e759-b3d7-4e22-b7eb-7403fb87da93 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:40.586888 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5e759-b3d7-4e22-b7eb-7403fb87da93 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:40.593828 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2c0ec-a40d-440c-8c06-86de03aa65a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:40.594536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2c0ec-a40d-440c-8c06-86de03aa65a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:40.594890 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2c0ec-a40d-440c-8c06-86de03aa65a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:40.595249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2c0ec-a40d-440c-8c06-86de03aa65a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:40.596144 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2c0ec-a40d-440c-8c06-86de03aa65a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:40.598519 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2c0ec-a40d-440c-8c06-86de03aa65a9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:40.599774 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 899/1777] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:40.600641 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28542384-1fdc-4eb6-b8f0-9494d4f56335 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:40.605097 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-28542384-1fdc-4eb6-b8f0-9494d4f56335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:40.635055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 879/1778] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:40 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:40.638198 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658ab38-4eba-41dc-82c7-27c55c70de9c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:40.640660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658ab38-4eba-41dc-82c7-27c55c70de9c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:40.640793 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658ab38-4eba-41dc-82c7-27c55c70de9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:40.690603 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d658ab38-4eba-41dc-82c7-27c55c70de9c None None] MFA Rules not processed for user `b6055476a66646e9b493e70a62931e71`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:40.692123 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d658ab38-4eba-41dc-82c7-27c55c70de9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:40.762578 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28542384-1fdc-4eb6-b8f0-9494d4f56335 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:40.788197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 880/1779] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:40.789149 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658ab38-4eba-41dc-82c7-27c55c70de9c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:40.887566 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea52d2-15e9-4b98-bd58-90d113a00ecd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:40.890133 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea52d2-15e9-4b98-bd58-90d113a00ecd service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:40.890525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea52d2-15e9-4b98-bd58-90d113a00ecd service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:40.890969 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea52d2-15e9-4b98-bd58-90d113a00ecd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:40.903297 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea52d2-15e9-4b98-bd58-90d113a00ecd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:40.906835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea52d2-15e9-4b98-bd58-90d113a00ecd service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:40.918601 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:40.931318 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 900/1780] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:40.932204 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea52d2-15e9-4b98-bd58-90d113a00ecd service nova] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:40.961995 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:40.976394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658ab38-4eba-41dc-82c7-27c55c70de9c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:40.990904 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3687e-3b8c-4fe4-983f-458ce05aceac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:40.992162 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3687e-3b8c-4fe4-983f-458ce05aceac admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:40.992458 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3687e-3b8c-4fe4-983f-458ce05aceac admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:40.992738 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3687e-3b8c-4fe4-983f-458ce05aceac admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:40.993450 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3687e-3b8c-4fe4-983f-458ce05aceac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:41.000264 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3687e-3b8c-4fe4-983f-458ce05aceac admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:41.064015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 881/1781] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:40 2022] POST /identity/v3/projects => generated 369 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:41.064015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3687e-3b8c-4fe4-983f-458ce05aceac admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:41.119051 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea52d2-15e9-4b98-bd58-90d113a00ecd service nova] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:41.190766 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3687e-3b8c-4fe4-983f-458ce05aceac admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:41.196713 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ab71d-0755-406e-9629-473fecc2fa56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:41.198447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ab71d-0755-406e-9629-473fecc2fa56 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:41.199386 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ab71d-0755-406e-9629-473fecc2fa56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:41.199817 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ab71d-0755-406e-9629-473fecc2fa56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:41.200705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ab71d-0755-406e-9629-473fecc2fa56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:41.203378 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ab71d-0755-406e-9629-473fecc2fa56 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:41.242322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 882/1782] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:41 2022] POST /identity/v3/projects => generated 379 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:41.243404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ab71d-0755-406e-9629-473fecc2fa56 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:41.271945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d2930-3a40-4739-b746-1cdb42284825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:41.272891 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d2930-3a40-4739-b746-1cdb42284825 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:41.273277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d2930-3a40-4739-b746-1cdb42284825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:41.273643 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d2930-3a40-4739-b746-1cdb42284825 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:41.274439 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d2930-3a40-4739-b746-1cdb42284825 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:41.276749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d2930-3a40-4739-b746-1cdb42284825 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:41.315034 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 901/1783] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:40 2022] POST /identity/v3/users => generated 287 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:41.318208 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d2930-3a40-4739-b746-1cdb42284825 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:41.335563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ab71d-0755-406e-9629-473fecc2fa56 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:41.343821 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d2930-3a40-4739-b746-1cdb42284825 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:41.349657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e930943-6e27-4656-a6a3-3603d27b5f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:41.350527 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e930943-6e27-4656-a6a3-3603d27b5f49 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:41.352267 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e930943-6e27-4656-a6a3-3603d27b5f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:41.352918 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc468f-be5a-4672-8e20-981585f90779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:41.353621 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e930943-6e27-4656-a6a3-3603d27b5f49 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:41.353798 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc468f-be5a-4672-8e20-981585f90779 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:41.354013 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc468f-be5a-4672-8e20-981585f90779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:41.354415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc468f-be5a-4672-8e20-981585f90779 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:41.355245 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc468f-be5a-4672-8e20-981585f90779 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:41.356151 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e930943-6e27-4656-a6a3-3603d27b5f49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:41.357257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc468f-be5a-4672-8e20-981585f90779 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:41.362419 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e930943-6e27-4656-a6a3-3603d27b5f49 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:41.397984 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 883/1784] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:41 2022] POST /identity/v3/users => generated 285 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:41.398843 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc468f-be5a-4672-8e20-981585f90779 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:41.406614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 902/1785] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:41 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:41.407691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e930943-6e27-4656-a6a3-3603d27b5f49 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:41.426394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc468f-be5a-4672-8e20-981585f90779 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:41.433590 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e930943-6e27-4656-a6a3-3603d27b5f49 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:41.437530 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccbc2e2-2254-4784-8324-a344fdc8befe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:41.438236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbc2e2-2254-4784-8324-a344fdc8befe admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:41.438525 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbc2e2-2254-4784-8324-a344fdc8befe admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:41.439283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbc2e2-2254-4784-8324-a344fdc8befe admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:41.441399 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccbc2e2-2254-4784-8324-a344fdc8befe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:41.445572 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccbc2e2-2254-4784-8324-a344fdc8befe admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:41.448842 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3746b-e89d-4147-9d7d-71a32cafdcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:41.449609 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3746b-e89d-4147-9d7d-71a32cafdcbb admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:41.450009 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3746b-e89d-4147-9d7d-71a32cafdcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:41.450353 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3746b-e89d-4147-9d7d-71a32cafdcbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:41.451161 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e3746b-e89d-4147-9d7d-71a32cafdcbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:41.453917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e3746b-e89d-4147-9d7d-71a32cafdcbb admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:41.469860 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 884/1786] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:41.476619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccbc2e2-2254-4784-8324-a344fdc8befe admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:41.483330 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 903/1787] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:41.484501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3746b-e89d-4147-9d7d-71a32cafdcbb admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:41.497084 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccbc2e2-2254-4784-8324-a344fdc8befe admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:41.506975 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3746b-e89d-4147-9d7d-71a32cafdcbb admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:41.514398 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca700ea9-774d-4163-b57e-602f01c0b85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:41.515543 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca700ea9-774d-4163-b57e-602f01c0b85f admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:41.516327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca700ea9-774d-4163-b57e-602f01c0b85f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:41.516910 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca700ea9-774d-4163-b57e-602f01c0b85f admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:41.517909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca700ea9-774d-4163-b57e-602f01c0b85f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:41.520029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17ebf5-f22a-4619-9994-7f791f80a404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:41.520940 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17ebf5-f22a-4619-9994-7f791f80a404 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:41.521415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17ebf5-f22a-4619-9994-7f791f80a404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:41.521764 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17ebf5-f22a-4619-9994-7f791f80a404 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:41.523109 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d17ebf5-f22a-4619-9994-7f791f80a404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:41.523358 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca700ea9-774d-4163-b57e-602f01c0b85f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:41.526107 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d17ebf5-f22a-4619-9994-7f791f80a404 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:41.544275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 885/1788] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:41.546107 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17ebf5-f22a-4619-9994-7f791f80a404 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:41.567655 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17ebf5-f22a-4619-9994-7f791f80a404 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:41.573550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be45d6-5ce7-4460-85a2-a37a841c8b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:41.574288 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be45d6-5ce7-4460-85a2-a37a841c8b8d admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:41.574577 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be45d6-5ce7-4460-85a2-a37a841c8b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:41.574853 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be45d6-5ce7-4460-85a2-a37a841c8b8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:41.575726 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75be45d6-5ce7-4460-85a2-a37a841c8b8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:41.576343 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 904/1789] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:41.578141 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca700ea9-774d-4163-b57e-602f01c0b85f admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:41.578400 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75be45d6-5ce7-4460-85a2-a37a841c8b8d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:41.580267 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ca700ea9-774d-4163-b57e-602f01c0b85f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:41.591863 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 886/1790] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:41.593121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be45d6-5ce7-4460-85a2-a37a841c8b8d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:41.601566 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca700ea9-774d-4163-b57e-602f01c0b85f admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:41.608681 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6131f7-8d04-4586-94cc-4eb17e2c212b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:41.609674 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6131f7-8d04-4586-94cc-4eb17e2c212b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:41.611307 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6131f7-8d04-4586-94cc-4eb17e2c212b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:41.611437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be45d6-5ce7-4460-85a2-a37a841c8b8d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:41.611594 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6131f7-8d04-4586-94cc-4eb17e2c212b admin admin] PATH_INFO: `/v3/projects/e564bcf94f444f2fbda896ebf85b6593` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:41.618257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702cdc6-242d-4f92-bc5b-00de9be68ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:41.619323 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702cdc6-242d-4f92-bc5b-00de9be68ed6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:41.619583 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6131f7-8d04-4586-94cc-4eb17e2c212b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e564bcf94f444f2fbda896ebf85b6593)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:41.619731 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702cdc6-242d-4f92-bc5b-00de9be68ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:41.619992 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702cdc6-242d-4f92-bc5b-00de9be68ed6 admin admin] PATH_INFO: `/v3/projects/0c54eaea9b7c40a38d4b18f3fff8cc70/users/1670d8f144b543cba319596c681c92f8/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:41.624226 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:41.626796 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6131f7-8d04-4586-94cc-4eb17e2c212b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:41.643688 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702cdc6-242d-4f92-bc5b-00de9be68ed6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c54eaea9b7c40a38d4b18f3fff8cc70, user_id=1670d8f144b543cba319596c681c92f8, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:41.646374 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702cdc6-242d-4f92-bc5b-00de9be68ed6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:41.666742 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 887/1791] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:41 2022] PUT /identity/v3/projects/0c54eaea9b7c40a38d4b18f3fff8cc70/users/1670d8f144b543cba319596c681c92f8/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:41.670323 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702cdc6-242d-4f92-bc5b-00de9be68ed6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:41.705304 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-2f6131f7-8d04-4586-94cc-4eb17e2c212b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e564bcf94f444f2fbda896ebf85b6593', 'request_id': 'req-2f6131f7-8d04-4586-94cc-4eb17e2c212b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:41.716090 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-2f6131f7-8d04-4586-94cc-4eb17e2c212b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e564bcf94f444f2fbda896ebf85b6593', 'request_id': 'req-2f6131f7-8d04-4586-94cc-4eb17e2c212b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:41.719734 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=88,keystone:INSERT=7,keystone:DELETE=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:41.726878 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-2f6131f7-8d04-4586-94cc-4eb17e2c212b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e564bcf94f444f2fbda896ebf85b6593', 'request_id': 'req-2f6131f7-8d04-4586-94cc-4eb17e2c212b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:41.741808 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-2f6131f7-8d04-4586-94cc-4eb17e2c212b admin admin] The token cache is being invalidate because project e564bcf94f444f2fbda896ebf85b6593 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:21:41.742251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-2f6131f7-8d04-4586-94cc-4eb17e2c212b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:41.743457 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-2f6131f7-8d04-4586-94cc-4eb17e2c212b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:41.745491 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 905/1792] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:21:41 2022] DELETE /identity/v3/projects/e564bcf94f444f2fbda896ebf85b6593 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:41.746673 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6131f7-8d04-4586-94cc-4eb17e2c212b admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:41.749153 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2f6131f7-8d04-4586-94cc-4eb17e2c212b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:41.773239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702cdc6-242d-4f92-bc5b-00de9be68ed6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:41.842789 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6131f7-8d04-4586-94cc-4eb17e2c212b admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:41.849035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c3f37-8bd1-439a-b9ff-e04180b95b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:41.850138 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c3f37-8bd1-439a-b9ff-e04180b95b12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:41.850515 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c3f37-8bd1-439a-b9ff-e04180b95b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:41.851409 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c3f37-8bd1-439a-b9ff-e04180b95b12 admin admin] PATH_INFO: `/v3/projects/f4af69b2482742988fe2a165a3a38e0b/users/a3808ed7e14a4263987eefbe866fbb56/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:41.865102 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa520798-1895-4a34-8400-facfd7f8cb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:41.866488 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa520798-1895-4a34-8400-facfd7f8cb71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:41.867061 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa520798-1895-4a34-8400-facfd7f8cb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:41.867639 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa520798-1895-4a34-8400-facfd7f8cb71 admin admin] PATH_INFO: `/v3/projects/1d7ba0d0d1e24ec488a585d52dc77a3c/users/f414b8819c2f429e9958bd989ba68123/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:41.874640 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c3f37-8bd1-439a-b9ff-e04180b95b12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4af69b2482742988fe2a165a3a38e0b, user_id=a3808ed7e14a4263987eefbe866fbb56, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:41.877863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c3f37-8bd1-439a-b9ff-e04180b95b12 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:41.890179 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa520798-1895-4a34-8400-facfd7f8cb71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d7ba0d0d1e24ec488a585d52dc77a3c, user_id=f414b8819c2f429e9958bd989ba68123, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:41.897098 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa520798-1895-4a34-8400-facfd7f8cb71 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:41.901670 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 906/1793] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:41 2022] PUT /identity/v3/projects/f4af69b2482742988fe2a165a3a38e0b/users/a3808ed7e14a4263987eefbe866fbb56/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:41.903488 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c3f37-8bd1-439a-b9ff-e04180b95b12 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:41.906134 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-973c3f37-8bd1-439a-b9ff-e04180b95b12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:41.937002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 888/1794] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:41 2022] PUT /identity/v3/projects/1d7ba0d0d1e24ec488a585d52dc77a3c/users/f414b8819c2f429e9958bd989ba68123/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:41.937002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa520798-1895-4a34-8400-facfd7f8cb71 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:41.937002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fa520798-1895-4a34-8400-facfd7f8cb71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:42.013470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c3f37-8bd1-439a-b9ff-e04180b95b12 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:42.039262 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa520798-1895-4a34-8400-facfd7f8cb71 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:42.049528 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1779f284-2768-4305-bc42-6f5d9108c8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:42.050456 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779f284-2768-4305-bc42-6f5d9108c8fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:42.050739 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779f284-2768-4305-bc42-6f5d9108c8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:42.051015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779f284-2768-4305-bc42-6f5d9108c8fb admin admin] PATH_INFO: `/v3/users/f09d7af261fe4e94a3f8e87a91180677` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:42.056318 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1779f284-2768-4305-bc42-6f5d9108c8fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f09d7af261fe4e94a3f8e87a91180677)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:42.062188 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1779f284-2768-4305-bc42-6f5d9108c8fb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:42.091274 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:42.118672 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c75e5e1-cc16-4f8f-a165-63729658330c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:42.119725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75e5e1-cc16-4f8f-a165-63729658330c admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:42.120469 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75e5e1-cc16-4f8f-a165-63729658330c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:42.120572 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75e5e1-cc16-4f8f-a165-63729658330c admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:42.121319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c75e5e1-cc16-4f8f-a165-63729658330c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:42.125037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c75e5e1-cc16-4f8f-a165-63729658330c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:42.142013 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 907/1795] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:42.148107 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfe67e-5176-47b3-8977-15af9debf312 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:42.148401 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfe67e-5176-47b3-8977-15af9debf312 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:42.148684 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfe67e-5176-47b3-8977-15af9debf312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:42.155797 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1779f284-2768-4305-bc42-6f5d9108c8fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f09d7af261fe4e94a3f8e87a91180677'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:42.158203 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1779f284-2768-4305-bc42-6f5d9108c8fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f09d7af261fe4e94a3f8e87a91180677'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:42.158849 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1779f284-2768-4305-bc42-6f5d9108c8fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f09d7af261fe4e94a3f8e87a91180677'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:42.183075 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1779f284-2768-4305-bc42-6f5d9108c8fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f09d7af261fe4e94a3f8e87a91180677'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:42.205888 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:42.210020 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-b9cfe67e-5176-47b3-8977-15af9debf312 None None] MFA Rules not processed for user `a3808ed7e14a4263987eefbe866fbb56`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:42.211379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b9cfe67e-5176-47b3-8977-15af9debf312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:42.215016 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1779f284-2768-4305-bc42-6f5d9108c8fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f09d7af261fe4e94a3f8e87a91180677'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:42.259251 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 889/1796] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:21:41 2022] DELETE /identity/v3/users/f09d7af261fe4e94a3f8e87a91180677 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:42.259251 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea9c6b-318a-4a4a-a5ce-34fcdb509709 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:42.259251 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea9c6b-318a-4a4a-a5ce-34fcdb509709 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:42.259251 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea9c6b-318a-4a4a-a5ce-34fcdb509709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:42.356137 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9eea9c6b-318a-4a4a-a5ce-34fcdb509709 None None] MFA Rules not processed for user `f414b8819c2f429e9958bd989ba68123`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:42.357452 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9eea9c6b-318a-4a4a-a5ce-34fcdb509709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:42.471874 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 908/1797] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:42 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:42.475704 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cfe67e-5176-47b3-8977-15af9debf312 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:42.478282 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b9cfe67e-5176-47b3-8977-15af9debf312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:42.503635 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cfe67e-5176-47b3-8977-15af9debf312 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:42.575666 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 890/1798] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:42.577559 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea9c6b-318a-4a4a-a5ce-34fcdb509709 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:42.579532 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9eea9c6b-318a-4a4a-a5ce-34fcdb509709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:42.742267 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d025c-9811-4f01-8cfa-26626644c878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:42.746513 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d025c-9811-4f01-8cfa-26626644c878 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:42.747099 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d025c-9811-4f01-8cfa-26626644c878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:42.748683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d025c-9811-4f01-8cfa-26626644c878 admin admin] PATH_INFO: `/v3/projects/0c54eaea9b7c40a38d4b18f3fff8cc70/users/1670d8f144b543cba319596c681c92f8/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:42.756232 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d025c-9811-4f01-8cfa-26626644c878 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c54eaea9b7c40a38d4b18f3fff8cc70, user_id=1670d8f144b543cba319596c681c92f8, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:42.756595 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea9c6b-318a-4a4a-a5ce-34fcdb509709 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:42.760295 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d025c-9811-4f01-8cfa-26626644c878 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:42.772130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e5a38-51df-4d21-a613-67d9bab0d0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:42.774249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e5a38-51df-4d21-a613-67d9bab0d0ad admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:42.774249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e5a38-51df-4d21-a613-67d9bab0d0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:42.774249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e5a38-51df-4d21-a613-67d9bab0d0ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:42.792152 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6e5a38-51df-4d21-a613-67d9bab0d0ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:42.800803 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6e5a38-51df-4d21-a613-67d9bab0d0ad admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:42.834817 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 909/1799] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:42 2022] PUT /identity/v3/projects/0c54eaea9b7c40a38d4b18f3fff8cc70/users/1670d8f144b543cba319596c681c92f8/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:42.837083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d025c-9811-4f01-8cfa-26626644c878 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:42.844007 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-331d025c-9811-4f01-8cfa-26626644c878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:42.920098 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 891/1800] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:42 2022] POST /identity/v3/projects => generated 371 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:42.928681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff608384-9b9b-4f20-8ffb-b4f4936ea7c1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:42.930952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff608384-9b9b-4f20-8ffb-b4f4936ea7c1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:42.931278 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff608384-9b9b-4f20-8ffb-b4f4936ea7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:43.019299 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:43.066184 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ff608384-9b9b-4f20-8ffb-b4f4936ea7c1 None None] MFA Rules not processed for user `1670d8f144b543cba319596c681c92f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:43.067623 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ff608384-9b9b-4f20-8ffb-b4f4936ea7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:43.125369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d025c-9811-4f01-8cfa-26626644c878 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:43.286453 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 892/1801] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:42 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:43.287240 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff608384-9b9b-4f20-8ffb-b4f4936ea7c1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:43.293249 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:43.412910 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd75a6-0e8d-471e-b60a-309ba452768e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:43.414928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd75a6-0e8d-471e-b60a-309ba452768e service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:43.414928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd75a6-0e8d-471e-b60a-309ba452768e service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:43.415402 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd75a6-0e8d-471e-b60a-309ba452768e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:43.444671 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff608384-9b9b-4f20-8ffb-b4f4936ea7c1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:43.460553 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7188f10-7a15-46fb-97a5-b7a347083bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:43.461359 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7188f10-7a15-46fb-97a5-b7a347083bc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:43.461710 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7188f10-7a15-46fb-97a5-b7a347083bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:43.462059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7188f10-7a15-46fb-97a5-b7a347083bc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:43.462923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7188f10-7a15-46fb-97a5-b7a347083bc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:43.465115 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7188f10-7a15-46fb-97a5-b7a347083bc7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:43.514901 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 893/1802] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:43 2022] POST /identity/v3/users => generated 292 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:43.516192 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7188f10-7a15-46fb-97a5-b7a347083bc7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:43.520919 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d7188f10-7a15-46fb-97a5-b7a347083bc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:43.545002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7188f10-7a15-46fb-97a5-b7a347083bc7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:43.555212 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd75a6-0e8d-471e-b60a-309ba452768e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:43.557585 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd75a6-0e8d-471e-b60a-309ba452768e service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:43.558652 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e6894-47b6-4f56-9334-123d9c2cc6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:43.559652 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e6894-47b6-4f56-9334-123d9c2cc6af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:43.560014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e6894-47b6-4f56-9334-123d9c2cc6af admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:43.560299 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e6894-47b6-4f56-9334-123d9c2cc6af admin admin] PATH_INFO: `/v3/projects/29c2010db6434f87a9111c40512bc4e5` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:43.562897 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e6894-47b6-4f56-9334-123d9c2cc6af admin admin] RBAC: Authorizing `identity:delete_project(project_id=29c2010db6434f87a9111c40512bc4e5)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:43.565022 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e6894-47b6-4f56-9334-123d9c2cc6af admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:43.577909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 910/1803] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 742 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:43.579262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd75a6-0e8d-471e-b60a-309ba452768e service nova] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:43.582343 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-debd75a6-0e8d-471e-b60a-309ba452768e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:43.673225 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-899e6894-47b6-4f56-9334-123d9c2cc6af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29c2010db6434f87a9111c40512bc4e5', 'request_id': 'req-899e6894-47b6-4f56-9334-123d9c2cc6af'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:43.685564 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-899e6894-47b6-4f56-9334-123d9c2cc6af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29c2010db6434f87a9111c40512bc4e5', 'request_id': 'req-899e6894-47b6-4f56-9334-123d9c2cc6af'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:43.702966 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-899e6894-47b6-4f56-9334-123d9c2cc6af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29c2010db6434f87a9111c40512bc4e5', 'request_id': 'req-899e6894-47b6-4f56-9334-123d9c2cc6af'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:43.729280 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd75a6-0e8d-471e-b60a-309ba452768e service nova] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:43.734098 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-899e6894-47b6-4f56-9334-123d9c2cc6af admin admin] The token cache is being invalidate because project 29c2010db6434f87a9111c40512bc4e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:21:43.735135 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-899e6894-47b6-4f56-9334-123d9c2cc6af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:43.738245 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-899e6894-47b6-4f56-9334-123d9c2cc6af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:43.746768 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 894/1804] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:21:43 2022] DELETE /identity/v3/projects/29c2010db6434f87a9111c40512bc4e5 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:43.749739 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e6894-47b6-4f56-9334-123d9c2cc6af admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:43.760217 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-899e6894-47b6-4f56-9334-123d9c2cc6af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:43.792683 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:43.966813 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e3ae13-11ff-479c-b49c-a774bc6b42e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:43.967905 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3ae13-11ff-479c-b49c-a774bc6b42e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:43.968813 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3ae13-11ff-479c-b49c-a774bc6b42e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:43.969110 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3ae13-11ff-479c-b49c-a774bc6b42e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:43.979357 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-30e3ae13-11ff-479c-b49c-a774bc6b42e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:44.045667 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e6894-47b6-4f56-9334-123d9c2cc6af admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:44.067286 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d17a3-9ab1-41a5-989d-cec387fb6190 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:44.072820 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d17a3-9ab1-41a5-989d-cec387fb6190 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:44.074395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d17a3-9ab1-41a5-989d-cec387fb6190 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:44.074702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d17a3-9ab1-41a5-989d-cec387fb6190 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:44.082696 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-db3d17a3-9ab1-41a5-989d-cec387fb6190 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:44.125006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:44.185184 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e3ae13-11ff-479c-b49c-a774bc6b42e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:44.202727 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e3ae13-11ff-479c-b49c-a774bc6b42e8 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:44.226886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 911/1805] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:43 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:44.228030 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e3ae13-11ff-479c-b49c-a774bc6b42e8 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:44.230489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-30e3ae13-11ff-479c-b49c-a774bc6b42e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:44.346330 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d17a3-9ab1-41a5-989d-cec387fb6190 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:44.349114 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d17a3-9ab1-41a5-989d-cec387fb6190 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:44.386114 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 895/1806] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:43 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:44.387826 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67f681-9f8b-4eb7-919b-03fb51db6bea None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:44.390346 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67f681-9f8b-4eb7-919b-03fb51db6bea None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:44.391894 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67f681-9f8b-4eb7-919b-03fb51db6bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:44.412114 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e3ae13-11ff-479c-b49c-a774bc6b42e8 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:44.432672 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b050f-e825-4403-aa3c-cd8353b2c9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:44.433500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b050f-e825-4403-aa3c-cd8353b2c9cb admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:44.433846 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b050f-e825-4403-aa3c-cd8353b2c9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:44.434190 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b050f-e825-4403-aa3c-cd8353b2c9cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:44.435153 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5a67f681-9f8b-4eb7-919b-03fb51db6bea None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:44.435339 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b050f-e825-4403-aa3c-cd8353b2c9cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:44.437371 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b050f-e825-4403-aa3c-cd8353b2c9cb admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:44.439078 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5a67f681-9f8b-4eb7-919b-03fb51db6bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:44.465364 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 912/1807] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:44.465364 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b050f-e825-4403-aa3c-cd8353b2c9cb admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:44.474089 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 896/1808] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:44.486157 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449179bc-61af-4b25-9baf-73a28627e44a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:44.486579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449179bc-61af-4b25-9baf-73a28627e44a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:44.486954 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449179bc-61af-4b25-9baf-73a28627e44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:44.498049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b050f-e825-4403-aa3c-cd8353b2c9cb admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:44.513631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec1a38-b44c-4251-b785-bf622cb3895b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:44.514678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec1a38-b44c-4251-b785-bf622cb3895b admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:44.515125 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec1a38-b44c-4251-b785-bf622cb3895b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:44.515441 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec1a38-b44c-4251-b785-bf622cb3895b admin admin] PATH_INFO: `/v3/projects/bae560d967ed495eb73233980a4648d4/users/b4fa144d0ab04d719735c02cc97eb6ff/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:44.542997 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-449179bc-61af-4b25-9baf-73a28627e44a None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:44.550721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-449179bc-61af-4b25-9baf-73a28627e44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:44.564496 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec1a38-b44c-4251-b785-bf622cb3895b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bae560d967ed495eb73233980a4648d4, user_id=b4fa144d0ab04d719735c02cc97eb6ff, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:44.573059 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec1a38-b44c-4251-b785-bf622cb3895b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:44.580693 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 897/1809] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:21:44.585405 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449179bc-61af-4b25-9baf-73a28627e44a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:44.606221 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449179bc-61af-4b25-9baf-73a28627e44a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:44.612376 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 913/1810] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:44 2022] PUT /identity/v3/projects/bae560d967ed495eb73233980a4648d4/users/b4fa144d0ab04d719735c02cc97eb6ff/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:44.613702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec1a38-b44c-4251-b785-bf622cb3895b admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:44.747202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda7cf2-e1e0-46d0-a578-ef7e9e84ab3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:44.750210 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda7cf2-e1e0-46d0-a578-ef7e9e84ab3f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:44.750732 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda7cf2-e1e0-46d0-a578-ef7e9e84ab3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:44.751626 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda7cf2-e1e0-46d0-a578-ef7e9e84ab3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:44.752501 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda7cf2-e1e0-46d0-a578-ef7e9e84ab3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:44.752828 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda7cf2-e1e0-46d0-a578-ef7e9e84ab3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:44.756538 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda7cf2-e1e0-46d0-a578-ef7e9e84ab3f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:44.799469 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 898/1811] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:21:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:44.800402 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda7cf2-e1e0-46d0-a578-ef7e9e84ab3f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:44.802970 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec1a38-b44c-4251-b785-bf622cb3895b admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:44.830229 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b3198-881e-4492-b959-a2e2bd4a9313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:44.832356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b3198-881e-4492-b959-a2e2bd4a9313 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:44.833216 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b3198-881e-4492-b959-a2e2bd4a9313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:44.833216 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b3198-881e-4492-b959-a2e2bd4a9313 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:44.841762 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b3198-881e-4492-b959-a2e2bd4a9313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:44.845921 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b3198-881e-4492-b959-a2e2bd4a9313 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:44.869404 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 914/1812] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:44.880708 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b3198-881e-4492-b959-a2e2bd4a9313 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:44.886584 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:44.908591 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda7cf2-e1e0-46d0-a578-ef7e9e84ab3f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:44.916887 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b3198-881e-4492-b959-a2e2bd4a9313 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:44.918873 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a068c16-8331-40ee-b8e4-30aeaac75469 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:44.919585 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a068c16-8331-40ee-b8e4-30aeaac75469 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:44.919874 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a068c16-8331-40ee-b8e4-30aeaac75469 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:44.920184 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a068c16-8331-40ee-b8e4-30aeaac75469 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:44.932027 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4ee28-e5ea-42c9-8792-a634ad59bd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:44.933925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4ee28-e5ea-42c9-8792-a634ad59bd5f admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:44.935233 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4ee28-e5ea-42c9-8792-a634ad59bd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:44.937099 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4ee28-e5ea-42c9-8792-a634ad59bd5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:44.939667 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4ee28-e5ea-42c9-8792-a634ad59bd5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:44.940972 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1a068c16-8331-40ee-b8e4-30aeaac75469 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:44.943300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4ee28-e5ea-42c9-8792-a634ad59bd5f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:45.055921 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 915/1813] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:44 2022] POST /identity/v3/projects => generated 373 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:45.055921 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4ee28-e5ea-42c9-8792-a634ad59bd5f admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:45.159308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a068c16-8331-40ee-b8e4-30aeaac75469 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:45.165444 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a068c16-8331-40ee-b8e4-30aeaac75469 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:45.185415 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:45.199985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 899/1814] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:45.201757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e666384-74ae-4028-b7cf-b10dd994125a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:45.202057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e666384-74ae-4028-b7cf-b10dd994125a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:45.202666 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e666384-74ae-4028-b7cf-b10dd994125a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:45.249422 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1e666384-74ae-4028-b7cf-b10dd994125a None None] MFA Rules not processed for user `6592aab6435242dca77b854a74ced4f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:45.253057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1e666384-74ae-4028-b7cf-b10dd994125a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:45.276316 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4ee28-e5ea-42c9-8792-a634ad59bd5f admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:45.285224 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff608c9-002d-4aa8-a6b9-f882a677acd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:45.287618 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff608c9-002d-4aa8-a6b9-f882a677acd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:45.288260 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff608c9-002d-4aa8-a6b9-f882a677acd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:45.288884 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff608c9-002d-4aa8-a6b9-f882a677acd9 admin admin] PATH_INFO: `/v3/projects/bae560d967ed495eb73233980a4648d4/users/b4fa144d0ab04d719735c02cc97eb6ff/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:45.293435 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff608c9-002d-4aa8-a6b9-f882a677acd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bae560d967ed495eb73233980a4648d4, user_id=b4fa144d0ab04d719735c02cc97eb6ff, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:45.297691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff608c9-002d-4aa8-a6b9-f882a677acd9 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:45.328246 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 916/1815] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:45 2022] PUT /identity/v3/projects/bae560d967ed495eb73233980a4648d4/users/b4fa144d0ab04d719735c02cc97eb6ff/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:45.329906 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff608c9-002d-4aa8-a6b9-f882a677acd9 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:45.438433 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff608c9-002d-4aa8-a6b9-f882a677acd9 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:45.445878 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835022d2-19b6-40ca-9ce1-b7b7cd60c1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:45.446676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835022d2-19b6-40ca-9ce1-b7b7cd60c1fe admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:45.446967 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835022d2-19b6-40ca-9ce1-b7b7cd60c1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:45.447355 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835022d2-19b6-40ca-9ce1-b7b7cd60c1fe admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:45.448174 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835022d2-19b6-40ca-9ce1-b7b7cd60c1fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:45.452474 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835022d2-19b6-40ca-9ce1-b7b7cd60c1fe admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:45.478562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 900/1816] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:45.482618 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbefdbe-f04e-4d81-a820-e7a14b0fd945 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:45.482965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbefdbe-f04e-4d81-a820-e7a14b0fd945 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:45.483268 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbefdbe-f04e-4d81-a820-e7a14b0fd945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:45.549492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 917/1817] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:45 2022] POST /identity/v3/users => generated 287 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:45.549492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835022d2-19b6-40ca-9ce1-b7b7cd60c1fe admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:45.586327 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ecbefdbe-f04e-4d81-a820-e7a14b0fd945 None None] MFA Rules not processed for user `b4fa144d0ab04d719735c02cc97eb6ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:45.590788 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ecbefdbe-f04e-4d81-a820-e7a14b0fd945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:45.599578 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835022d2-19b6-40ca-9ce1-b7b7cd60c1fe admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:45.621069 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4c5e4-36d4-4767-b236-f12e16249196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:45.621768 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4c5e4-36d4-4767-b236-f12e16249196 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:45.622059 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4c5e4-36d4-4767-b236-f12e16249196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:45.626635 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4c5e4-36d4-4767-b236-f12e16249196 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:45.632570 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4c5e4-36d4-4767-b236-f12e16249196 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:45.635096 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4c5e4-36d4-4767-b236-f12e16249196 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:45.672772 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 918/1818] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:45.674950 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4c5e4-36d4-4767-b236-f12e16249196 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:45.704914 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4c5e4-36d4-4767-b236-f12e16249196 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:45.736858 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef72824-0cb9-4249-9afc-7ac5cb0fba9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:45.739626 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef72824-0cb9-4249-9afc-7ac5cb0fba9c admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:45.740067 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef72824-0cb9-4249-9afc-7ac5cb0fba9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:45.740497 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef72824-0cb9-4249-9afc-7ac5cb0fba9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:45.744413 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef72824-0cb9-4249-9afc-7ac5cb0fba9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:45.753108 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef72824-0cb9-4249-9afc-7ac5cb0fba9c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:45.803162 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 919/1819] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:45.804031 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 901/1820] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:45 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:45.810115 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef72824-0cb9-4249-9afc-7ac5cb0fba9c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:45.837643 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef72824-0cb9-4249-9afc-7ac5cb0fba9c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:45.847780 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37576b9-44a4-4dc1-9c21-565e4eb9d2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:45.849052 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37576b9-44a4-4dc1-9c21-565e4eb9d2fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:45.849484 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37576b9-44a4-4dc1-9c21-565e4eb9d2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:45.850127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37576b9-44a4-4dc1-9c21-565e4eb9d2fa admin admin] PATH_INFO: `/v3/projects/2a8e39e02778437dbc094914785b7681/users/212ebdcd4c584bebaaa83d814d8069b9/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:45.878115 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37576b9-44a4-4dc1-9c21-565e4eb9d2fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a8e39e02778437dbc094914785b7681, user_id=212ebdcd4c584bebaaa83d814d8069b9, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:45.886383 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37576b9-44a4-4dc1-9c21-565e4eb9d2fa admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:45.926826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 920/1821] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:45 2022] PUT /identity/v3/projects/2a8e39e02778437dbc094914785b7681/users/212ebdcd4c584bebaaa83d814d8069b9/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:45.931083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7eaf81-951d-4bac-b868-f57a80062811 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:45.931383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7eaf81-951d-4bac-b868-f57a80062811 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:45.931724 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7eaf81-951d-4bac-b868-f57a80062811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:45.978340 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37576b9-44a4-4dc1-9c21-565e4eb9d2fa admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:45.980956 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d37576b9-44a4-4dc1-9c21-565e4eb9d2fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:45.983483 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9b7eaf81-951d-4bac-b868-f57a80062811 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:45.984708 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9b7eaf81-951d-4bac-b868-f57a80062811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:46.038303 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:46.159532 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37576b9-44a4-4dc1-9c21-565e4eb9d2fa admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:46.167368 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 902/1822] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:46.179623 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b9eb2-709a-452d-9884-f89a9a9b8a31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:46.180632 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b9eb2-709a-452d-9884-f89a9a9b8a31 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:46.181091 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b9eb2-709a-452d-9884-f89a9a9b8a31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:46.181598 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b9eb2-709a-452d-9884-f89a9a9b8a31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:46.208364 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:46.288750 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7eaf81-951d-4bac-b868-f57a80062811 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:46.293753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9b7eaf81-951d-4bac-b868-f57a80062811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:46.357353 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b9eb2-709a-452d-9884-f89a9a9b8a31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:46.361478 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b9eb2-709a-452d-9884-f89a9a9b8a31 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:46.398105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 921/1823] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:45 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:46.442686 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7eaf81-951d-4bac-b868-f57a80062811 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:46.457230 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c85a0-a3f4-48cb-a2f7-6b002fdc9856 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:46.460362 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c85a0-a3f4-48cb-a2f7-6b002fdc9856 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:46.460670 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c85a0-a3f4-48cb-a2f7-6b002fdc9856 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:46.460951 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c85a0-a3f4-48cb-a2f7-6b002fdc9856 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:46.490840 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c85a0-a3f4-48cb-a2f7-6b002fdc9856 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:46.501739 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c85a0-a3f4-48cb-a2f7-6b002fdc9856 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:46.525476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 903/1824] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:21:47.149274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a37e42-c4a9-449d-9ffe-90f205e1664e None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:47.149874 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a37e42-c4a9-449d-9ffe-90f205e1664e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:47.149874 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a37e42-c4a9-449d-9ffe-90f205e1664e None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:47.151738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 922/1825] 10.222.0.81 () {54 vars in 912 bytes} [Tue Jun 7 14:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:47.174326 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149748b-d3e6-4214-a333-8b64a9319a8c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:47.174326 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149748b-d3e6-4214-a333-8b64a9319a8c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:47.174842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149748b-d3e6-4214-a333-8b64a9319a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:47.205670 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5149748b-d3e6-4214-a333-8b64a9319a8c None None] MFA Rules not processed for user `a6a08eaecbcd41228571739d8387882a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:47.207589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5149748b-d3e6-4214-a333-8b64a9319a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:47.217387 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 904/1826] 10.222.0.81 () {58 vars in 1009 bytes} [Tue Jun 7 14:21:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:47.252268 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a37e42-c4a9-449d-9ffe-90f205e1664e None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:47.273264 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a37e42-c4a9-449d-9ffe-90f205e1664e None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:47.291829 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228bfdd-91f7-4767-b67f-a296e103c98f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:47.292707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228bfdd-91f7-4767-b67f-a296e103c98f service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:47.295210 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228bfdd-91f7-4767-b67f-a296e103c98f service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:47.295810 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228bfdd-91f7-4767-b67f-a296e103c98f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:47.319837 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a228bfdd-91f7-4767-b67f-a296e103c98f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:47.321612 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a228bfdd-91f7-4767-b67f-a296e103c98f service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:47.335828 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 923/1827] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:47.345336 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149748b-d3e6-4214-a333-8b64a9319a8c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:47.366635 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149748b-d3e6-4214-a333-8b64a9319a8c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:47.372236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea94758a-d1ad-460d-a7f5-04ae5f622696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:47.374022 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94758a-d1ad-460d-a7f5-04ae5f622696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:47.374484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94758a-d1ad-460d-a7f5-04ae5f622696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:47.374964 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94758a-d1ad-460d-a7f5-04ae5f622696 admin admin] PATH_INFO: `/v3/users/a3808ed7e14a4263987eefbe866fbb56` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:47.376311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea94758a-d1ad-460d-a7f5-04ae5f622696 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3808ed7e14a4263987eefbe866fbb56)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:47.378723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea94758a-d1ad-460d-a7f5-04ae5f622696 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:47.441659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ea94758a-d1ad-460d-a7f5-04ae5f622696 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3808ed7e14a4263987eefbe866fbb56'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:47.442474 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ea94758a-d1ad-460d-a7f5-04ae5f622696 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3808ed7e14a4263987eefbe866fbb56'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:47.443844 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ea94758a-d1ad-460d-a7f5-04ae5f622696 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3808ed7e14a4263987eefbe866fbb56'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:47.469684 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ea94758a-d1ad-460d-a7f5-04ae5f622696 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3808ed7e14a4263987eefbe866fbb56'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:47.498619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ea94758a-d1ad-460d-a7f5-04ae5f622696 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3808ed7e14a4263987eefbe866fbb56'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:47.526328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 905/1828] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:21:47 2022] DELETE /identity/v3/users/a3808ed7e14a4263987eefbe866fbb56 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:47.528732 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120ff01-ed8f-4131-a2d8-dff38d85c77f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:47.529244 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120ff01-ed8f-4131-a2d8-dff38d85c77f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:47.529747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120ff01-ed8f-4131-a2d8-dff38d85c77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:47.550716 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-3120ff01-ed8f-4131-a2d8-dff38d85c77f None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:47.552274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3120ff01-ed8f-4131-a2d8-dff38d85c77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:47.675783 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 924/1829] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:47.689505 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea94758a-d1ad-460d-a7f5-04ae5f622696 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:47.691582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ea94758a-d1ad-460d-a7f5-04ae5f622696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:47.741594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:47.775750 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea94758a-d1ad-460d-a7f5-04ae5f622696 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:47.780163 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9124c3cb-ddad-4135-aa71-caf66e1c9ae4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:47.781029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124c3cb-ddad-4135-aa71-caf66e1c9ae4 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:47.781383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124c3cb-ddad-4135-aa71-caf66e1c9ae4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:47.781749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124c3cb-ddad-4135-aa71-caf66e1c9ae4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:47.793327 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9124c3cb-ddad-4135-aa71-caf66e1c9ae4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:47.795032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9124c3cb-ddad-4135-aa71-caf66e1c9ae4 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:47.806527 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 906/1830] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:48.322487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65036d01-20f2-42f3-80de-9cbaa58089ff None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:48.323015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65036d01-20f2-42f3-80de-9cbaa58089ff None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:48.323092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65036d01-20f2-42f3-80de-9cbaa58089ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:48.358828 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-65036d01-20f2-42f3-80de-9cbaa58089ff None None] MFA Rules not processed for user `212ebdcd4c584bebaaa83d814d8069b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:48.360252 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-65036d01-20f2-42f3-80de-9cbaa58089ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:48.446443 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 925/1831] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:48.454430 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9124c3cb-ddad-4135-aa71-caf66e1c9ae4 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:48.454726 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65036d01-20f2-42f3-80de-9cbaa58089ff None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:48.456169 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-65036d01-20f2-42f3-80de-9cbaa58089ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:48.456284 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9124c3cb-ddad-4135-aa71-caf66e1c9ae4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:48.470940 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65036d01-20f2-42f3-80de-9cbaa58089ff None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:48.477932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb20d1-1566-4812-8478-f7e29d7cd138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:48.478143 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9124c3cb-ddad-4135-aa71-caf66e1c9ae4 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:48.479110 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb20d1-1566-4812-8478-f7e29d7cd138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:48.479429 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb20d1-1566-4812-8478-f7e29d7cd138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:48.479699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb20d1-1566-4812-8478-f7e29d7cd138 admin admin] PATH_INFO: `/v3/projects/f4af69b2482742988fe2a165a3a38e0b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:48.480919 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb20d1-1566-4812-8478-f7e29d7cd138 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4af69b2482742988fe2a165a3a38e0b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:48.482907 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb20d1-1566-4812-8478-f7e29d7cd138 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:48.490371 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe7c709-10c6-4728-a173-a1ab8821279d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:48.491177 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7c709-10c6-4728-a173-a1ab8821279d admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:48.491506 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7c709-10c6-4728-a173-a1ab8821279d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:48.492264 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7c709-10c6-4728-a173-a1ab8821279d admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:48.492980 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe7c709-10c6-4728-a173-a1ab8821279d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:48.494874 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe7c709-10c6-4728-a173-a1ab8821279d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:48.510336 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:48.521520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 907/1832] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:48 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:21:48.525377 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe7c709-10c6-4728-a173-a1ab8821279d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:48.537193 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d8fb20d1-1566-4812-8478-f7e29d7cd138 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4af69b2482742988fe2a165a3a38e0b', 'request_id': 'req-d8fb20d1-1566-4812-8478-f7e29d7cd138'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:48.550039 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d8fb20d1-1566-4812-8478-f7e29d7cd138 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4af69b2482742988fe2a165a3a38e0b', 'request_id': 'req-d8fb20d1-1566-4812-8478-f7e29d7cd138'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:48.558409 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d8fb20d1-1566-4812-8478-f7e29d7cd138 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4af69b2482742988fe2a165a3a38e0b', 'request_id': 'req-d8fb20d1-1566-4812-8478-f7e29d7cd138'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:48.572589 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d8fb20d1-1566-4812-8478-f7e29d7cd138 admin admin] The token cache is being invalidate because project f4af69b2482742988fe2a165a3a38e0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:21:48.572963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d8fb20d1-1566-4812-8478-f7e29d7cd138 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:48.573717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d8fb20d1-1566-4812-8478-f7e29d7cd138 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:48.575400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 926/1833] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:21:48 2022] DELETE /identity/v3/projects/f4af69b2482742988fe2a165a3a38e0b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:21:48.580025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb20d1-1566-4812-8478-f7e29d7cd138 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:48.582113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d8fb20d1-1566-4812-8478-f7e29d7cd138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:48.597403 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe7c709-10c6-4728-a173-a1ab8821279d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:48.659613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb20d1-1566-4812-8478-f7e29d7cd138 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:48.664397 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785b7d6-c621-4577-ad9a-caae03523fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:48.665149 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785b7d6-c621-4577-ad9a-caae03523fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:48.665479 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e0442-5b8f-4bcc-864f-2f0c1a632d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:48.665681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785b7d6-c621-4577-ad9a-caae03523fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:48.666018 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785b7d6-c621-4577-ad9a-caae03523fc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:48.666356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e0442-5b8f-4bcc-864f-2f0c1a632d89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:48.666638 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e0442-5b8f-4bcc-864f-2f0c1a632d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:48.666895 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785b7d6-c621-4577-ad9a-caae03523fc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:48.666998 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e0442-5b8f-4bcc-864f-2f0c1a632d89 admin admin] PATH_INFO: `/v3/users/b4fa144d0ab04d719735c02cc97eb6ff` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:48.668135 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63e0442-5b8f-4bcc-864f-2f0c1a632d89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4fa144d0ab04d719735c02cc97eb6ff)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:48.669151 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785b7d6-c621-4577-ad9a-caae03523fc7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:48.670108 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63e0442-5b8f-4bcc-864f-2f0c1a632d89 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:48.694247 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 908/1834] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:48 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:21:48.695928 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785b7d6-c621-4577-ad9a-caae03523fc7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:48.697882 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1785b7d6-c621-4577-ad9a-caae03523fc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:48.712404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785b7d6-c621-4577-ad9a-caae03523fc7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:48.718089 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de240ca-dafc-4bb2-9ad8-2104f674fbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:48.718798 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de240ca-dafc-4bb2-9ad8-2104f674fbc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:48.719496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de240ca-dafc-4bb2-9ad8-2104f674fbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:48.719989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de240ca-dafc-4bb2-9ad8-2104f674fbc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:48.720737 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de240ca-dafc-4bb2-9ad8-2104f674fbc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:48.722342 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de240ca-dafc-4bb2-9ad8-2104f674fbc8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:48.730976 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a63e0442-5b8f-4bcc-864f-2f0c1a632d89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4fa144d0ab04d719735c02cc97eb6ff'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:48.742681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 909/1835] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:48.743683 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de240ca-dafc-4bb2-9ad8-2104f674fbc8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:48.750838 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a63e0442-5b8f-4bcc-864f-2f0c1a632d89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4fa144d0ab04d719735c02cc97eb6ff'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:48.752112 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a63e0442-5b8f-4bcc-864f-2f0c1a632d89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4fa144d0ab04d719735c02cc97eb6ff'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:48.752842 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a63e0442-5b8f-4bcc-864f-2f0c1a632d89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4fa144d0ab04d719735c02cc97eb6ff'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:48.755485 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de240ca-dafc-4bb2-9ad8-2104f674fbc8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:48.760955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd387fff-0cda-413b-80bd-c1665cbede2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:48.761205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a63e0442-5b8f-4bcc-864f-2f0c1a632d89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4fa144d0ab04d719735c02cc97eb6ff'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:48.761980 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd387fff-0cda-413b-80bd-c1665cbede2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:48.762449 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd387fff-0cda-413b-80bd-c1665cbede2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:48.762829 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd387fff-0cda-413b-80bd-c1665cbede2e admin admin] PATH_INFO: `/v3/projects/c224108509ea40bda53e027705732cd4/users/e0af5a5eb22f4eb28415c40abb03d8ed/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:48.786410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 927/1836] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:21:48 2022] DELETE /identity/v3/users/b4fa144d0ab04d719735c02cc97eb6ff => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:21:48.786580 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd387fff-0cda-413b-80bd-c1665cbede2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c224108509ea40bda53e027705732cd4, user_id=e0af5a5eb22f4eb28415c40abb03d8ed, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:48.789825 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd387fff-0cda-413b-80bd-c1665cbede2e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:48.809421 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 910/1837] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:48 2022] PUT /identity/v3/projects/c224108509ea40bda53e027705732cd4/users/e0af5a5eb22f4eb28415c40abb03d8ed/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:48.813776 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e0442-5b8f-4bcc-864f-2f0c1a632d89 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:48.815318 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a63e0442-5b8f-4bcc-864f-2f0c1a632d89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:48.853518 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:48.887775 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e0442-5b8f-4bcc-864f-2f0c1a632d89 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:48.953662 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb2dc0-9b1e-4054-b5b8-4b51c048e0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:48.954339 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb2dc0-9b1e-4054-b5b8-4b51c048e0ec admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:48.954622 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb2dc0-9b1e-4054-b5b8-4b51c048e0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:48.955020 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb2dc0-9b1e-4054-b5b8-4b51c048e0ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:48.955806 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeb2dc0-9b1e-4054-b5b8-4b51c048e0ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:48.957406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeb2dc0-9b1e-4054-b5b8-4b51c048e0ec admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:48.969723 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 928/1838] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:48.973853 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd387fff-0cda-413b-80bd-c1665cbede2e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:48.986557 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd387fff-0cda-413b-80bd-c1665cbede2e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:48.993922 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa9209-2016-4a18-85fe-5d4527065a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:48.995253 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa9209-2016-4a18-85fe-5d4527065a88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:48.995776 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa9209-2016-4a18-85fe-5d4527065a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:48.996265 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa9209-2016-4a18-85fe-5d4527065a88 admin admin] PATH_INFO: `/v3/projects/c224108509ea40bda53e027705732cd4/users/e0af5a5eb22f4eb28415c40abb03d8ed/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:48.999990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa9209-2016-4a18-85fe-5d4527065a88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c224108509ea40bda53e027705732cd4, user_id=e0af5a5eb22f4eb28415c40abb03d8ed, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:49.002429 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa9209-2016-4a18-85fe-5d4527065a88 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:49.011079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:49.016475 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 911/1839] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:48 2022] PUT /identity/v3/projects/c224108509ea40bda53e027705732cd4/users/e0af5a5eb22f4eb28415c40abb03d8ed/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:21:49.182328 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb2dc0-9b1e-4054-b5b8-4b51c048e0ec admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:49.184178 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-dfeb2dc0-9b1e-4054-b5b8-4b51c048e0ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:49.197949 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb2dc0-9b1e-4054-b5b8-4b51c048e0ec admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:49.281193 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce29ec6-0500-4f6d-b747-f6a058ed6d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:49.281807 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce29ec6-0500-4f6d-b747-f6a058ed6d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:49.282136 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce29ec6-0500-4f6d-b747-f6a058ed6d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:49.282426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce29ec6-0500-4f6d-b747-f6a058ed6d80 admin admin] PATH_INFO: `/v3/projects/bae560d967ed495eb73233980a4648d4` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:49.283762 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce29ec6-0500-4f6d-b747-f6a058ed6d80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bae560d967ed495eb73233980a4648d4)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:49.285985 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce29ec6-0500-4f6d-b747-f6a058ed6d80 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:49.338456 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1ce29ec6-0500-4f6d-b747-f6a058ed6d80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bae560d967ed495eb73233980a4648d4', 'request_id': 'req-1ce29ec6-0500-4f6d-b747-f6a058ed6d80'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:49.346506 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1ce29ec6-0500-4f6d-b747-f6a058ed6d80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bae560d967ed495eb73233980a4648d4', 'request_id': 'req-1ce29ec6-0500-4f6d-b747-f6a058ed6d80'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:49.353811 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1ce29ec6-0500-4f6d-b747-f6a058ed6d80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bae560d967ed495eb73233980a4648d4', 'request_id': 'req-1ce29ec6-0500-4f6d-b747-f6a058ed6d80'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:49.368379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1ce29ec6-0500-4f6d-b747-f6a058ed6d80 admin admin] The token cache is being invalidate because project bae560d967ed495eb73233980a4648d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:21:49.368778 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1ce29ec6-0500-4f6d-b747-f6a058ed6d80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:49.370213 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1ce29ec6-0500-4f6d-b747-f6a058ed6d80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:49.372789 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 929/1840] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:21:49 2022] DELETE /identity/v3/projects/bae560d967ed495eb73233980a4648d4 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:49.378458 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d47bb-cdd9-4c32-ac1c-a1504d06e6fb None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:49.378703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d47bb-cdd9-4c32-ac1c-a1504d06e6fb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:49.378944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d47bb-cdd9-4c32-ac1c-a1504d06e6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:49.398862 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-011d47bb-cdd9-4c32-ac1c-a1504d06e6fb None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:49.400084 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-011d47bb-cdd9-4c32-ac1c-a1504d06e6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:49.464311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 912/1841] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:49.472645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a5d0f-aa74-48b8-9d15-44ff60ae506e None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:49.472934 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a5d0f-aa74-48b8-9d15-44ff60ae506e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:49.473212 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a5d0f-aa74-48b8-9d15-44ff60ae506e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:49.482278 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:49.496191 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-865a5d0f-aa74-48b8-9d15-44ff60ae506e None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:49.497657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-865a5d0f-aa74-48b8-9d15-44ff60ae506e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:49.514810 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 930/1842] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:49.519082 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d47bb-cdd9-4c32-ac1c-a1504d06e6fb None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:49.529813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d47bb-cdd9-4c32-ac1c-a1504d06e6fb None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:49.534473 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752bd2d-84c2-41e3-8ca9-31c82f2cb0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:49.535664 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752bd2d-84c2-41e3-8ca9-31c82f2cb0ba admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:49.535983 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752bd2d-84c2-41e3-8ca9-31c82f2cb0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:49.536237 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752bd2d-84c2-41e3-8ca9-31c82f2cb0ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:49.536961 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752bd2d-84c2-41e3-8ca9-31c82f2cb0ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:49.537262 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752bd2d-84c2-41e3-8ca9-31c82f2cb0ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:49.538903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752bd2d-84c2-41e3-8ca9-31c82f2cb0ba admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:49.555488 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 913/1843] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:49.564485 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a5d0f-aa74-48b8-9d15-44ff60ae506e None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:49.582703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a5d0f-aa74-48b8-9d15-44ff60ae506e None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:49.588222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ad83e-af66-4973-9e6d-988edfd83bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:49.588888 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ad83e-af66-4973-9e6d-988edfd83bbd admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:49.589167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ad83e-af66-4973-9e6d-988edfd83bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:49.589439 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ad83e-af66-4973-9e6d-988edfd83bbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:49.590139 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61ad83e-af66-4973-9e6d-988edfd83bbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:49.592224 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61ad83e-af66-4973-9e6d-988edfd83bbd admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:49.625658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 931/1844] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:49 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:49.626876 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752bd2d-84c2-41e3-8ca9-31c82f2cb0ba admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:49.637112 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752bd2d-84c2-41e3-8ca9-31c82f2cb0ba admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:49.714455 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01619c-638f-4b0e-b9c4-9f710d0a1df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:49.715129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01619c-638f-4b0e-b9c4-9f710d0a1df5 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:49.715450 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01619c-638f-4b0e-b9c4-9f710d0a1df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:49.715733 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01619c-638f-4b0e-b9c4-9f710d0a1df5 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:49.716628 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd01619c-638f-4b0e-b9c4-9f710d0a1df5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:49.719285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd01619c-638f-4b0e-b9c4-9f710d0a1df5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:49.744829 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 914/1845] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:49 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:21:49.749328 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ad83e-af66-4973-9e6d-988edfd83bbd admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:49.759420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ad83e-af66-4973-9e6d-988edfd83bbd admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:49.764625 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8ecac-da41-4707-9add-c20f2106ec60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:49.765670 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8ecac-da41-4707-9add-c20f2106ec60 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:49.765957 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8ecac-da41-4707-9add-c20f2106ec60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:49.766233 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8ecac-da41-4707-9add-c20f2106ec60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:49.766962 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f8ecac-da41-4707-9add-c20f2106ec60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:49.768621 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f8ecac-da41-4707-9add-c20f2106ec60 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:49.782039 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 932/1846] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:49.784154 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01619c-638f-4b0e-b9c4-9f710d0a1df5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:49.794964 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01619c-638f-4b0e-b9c4-9f710d0a1df5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:49.811474 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983264e-ac42-423b-affc-d4e61c261419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:49.811769 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983264e-ac42-423b-affc-d4e61c261419 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:49.812019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983264e-ac42-423b-affc-d4e61c261419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:49.812298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983264e-ac42-423b-affc-d4e61c261419 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:49.813037 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3983264e-ac42-423b-affc-d4e61c261419 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:49.814643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3983264e-ac42-423b-affc-d4e61c261419 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:49.916413 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 915/1847] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:49.918248 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8ecac-da41-4707-9add-c20f2106ec60 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:49.932391 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8ecac-da41-4707-9add-c20f2106ec60 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:49.939600 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33888a5d-8aa1-4665-b82e-fa47ee688b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:49.941183 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33888a5d-8aa1-4665-b82e-fa47ee688b3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:49.941680 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33888a5d-8aa1-4665-b82e-fa47ee688b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:49.942135 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33888a5d-8aa1-4665-b82e-fa47ee688b3c admin admin] PATH_INFO: `/v3/projects/86764c2a9c5b4f4b96ea56618f439e08/users/7f5966e6a1134057b77d5b1811582d4d/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:49.961817 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33888a5d-8aa1-4665-b82e-fa47ee688b3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=86764c2a9c5b4f4b96ea56618f439e08, user_id=7f5966e6a1134057b77d5b1811582d4d, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:49.965178 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33888a5d-8aa1-4665-b82e-fa47ee688b3c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:49.989076 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 933/1848] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:49 2022] PUT /identity/v3/projects/86764c2a9c5b4f4b96ea56618f439e08/users/7f5966e6a1134057b77d5b1811582d4d/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:50.000941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057924d-8366-4ec2-9196-6ae8f0fc974a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:50.001243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057924d-8366-4ec2-9196-6ae8f0fc974a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:50.001496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057924d-8366-4ec2-9196-6ae8f0fc974a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:50.009993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095f6b2-256a-4ab3-b638-7fb8eb43826c None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:50.012279 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095f6b2-256a-4ab3-b638-7fb8eb43826c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:50.012279 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095f6b2-256a-4ab3-b638-7fb8eb43826c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:50.044487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c057924d-8366-4ec2-9196-6ae8f0fc974a None None] MFA Rules not processed for user `7f5966e6a1134057b77d5b1811582d4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:50.046100 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c057924d-8366-4ec2-9196-6ae8f0fc974a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:50.064101 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-3095f6b2-256a-4ab3-b638-7fb8eb43826c None None] MFA Rules not processed for user `e0af5a5eb22f4eb28415c40abb03d8ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:50.065500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3095f6b2-256a-4ab3-b638-7fb8eb43826c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:50.229244 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:50.253289 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 916/1849] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:50.260291 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057924d-8366-4ec2-9196-6ae8f0fc974a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:50.293537 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 934/1850] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:50 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:50.303753 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095f6b2-256a-4ab3-b638-7fb8eb43826c None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:50.306375 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3095f6b2-256a-4ab3-b638-7fb8eb43826c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:50.392034 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095f6b2-256a-4ab3-b638-7fb8eb43826c None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:50.396596 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88318fb9-3085-482d-ab4d-b0b6b0ccad9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:50.397452 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88318fb9-3085-482d-ab4d-b0b6b0ccad9c service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:50.398118 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88318fb9-3085-482d-ab4d-b0b6b0ccad9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:50.398498 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88318fb9-3085-482d-ab4d-b0b6b0ccad9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:50.401490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-88318fb9-3085-482d-ab4d-b0b6b0ccad9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:50.457363 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057924d-8366-4ec2-9196-6ae8f0fc974a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:50.473525 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19966b-f932-4403-9ea3-cf4fe00266ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:50.474539 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19966b-f932-4403-9ea3-cf4fe00266ba admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:50.475099 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19966b-f932-4403-9ea3-cf4fe00266ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:50.475390 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:50.475889 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19966b-f932-4403-9ea3-cf4fe00266ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:50.476607 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19966b-f932-4403-9ea3-cf4fe00266ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:50.480438 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19966b-f932-4403-9ea3-cf4fe00266ba admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:50.500319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88318fb9-3085-482d-ab4d-b0b6b0ccad9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:50.501889 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88318fb9-3085-482d-ab4d-b0b6b0ccad9c service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:50.513799 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 935/1851] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:21:50.552014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 917/1852] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:50 2022] POST /identity/v3/projects => generated 379 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:21:50.556383 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88318fb9-3085-482d-ab4d-b0b6b0ccad9c service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:50.573123 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88318fb9-3085-482d-ab4d-b0b6b0ccad9c service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:50.657303 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff5b31-5c83-4f8d-8a2e-0a7e14071564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:50.658323 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff5b31-5c83-4f8d-8a2e-0a7e14071564 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:50.658640 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff5b31-5c83-4f8d-8a2e-0a7e14071564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:50.659762 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff5b31-5c83-4f8d-8a2e-0a7e14071564 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:50.660747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff5b31-5c83-4f8d-8a2e-0a7e14071564 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:50.664024 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff5b31-5c83-4f8d-8a2e-0a7e14071564 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:50.701731 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 936/1853] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:50 2022] POST /identity/v3/users => generated 296 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:21:50.705578 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19966b-f932-4403-9ea3-cf4fe00266ba admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:50.716020 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19966b-f932-4403-9ea3-cf4fe00266ba admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:50.722099 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8bd8f-7f56-4c4d-9972-f3e1b5f29ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:50.723036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8bd8f-7f56-4c4d-9972-f3e1b5f29ab1 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:50.723628 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8bd8f-7f56-4c4d-9972-f3e1b5f29ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:50.724081 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8bd8f-7f56-4c4d-9972-f3e1b5f29ab1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:50.724987 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8bd8f-7f56-4c4d-9972-f3e1b5f29ab1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:50.727026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8bd8f-7f56-4c4d-9972-f3e1b5f29ab1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:50.744780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 918/1854] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:50.749167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff5b31-5c83-4f8d-8a2e-0a7e14071564 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:50.760821 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff5b31-5c83-4f8d-8a2e-0a7e14071564 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:50.774784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce192ba-3017-4b22-9dbb-e4278ee18f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:50.775862 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce192ba-3017-4b22-9dbb-e4278ee18f28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:50.776197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce192ba-3017-4b22-9dbb-e4278ee18f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:50.776690 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce192ba-3017-4b22-9dbb-e4278ee18f28 admin admin] PATH_INFO: `/v3/projects/a054de6c24364e7abc26f003185755ac/users/436f3c7565f6486ebbaffaf6aec7df23/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:50.824282 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce192ba-3017-4b22-9dbb-e4278ee18f28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a054de6c24364e7abc26f003185755ac, user_id=436f3c7565f6486ebbaffaf6aec7df23, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:50.837155 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce192ba-3017-4b22-9dbb-e4278ee18f28 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:50.863275 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 937/1855] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:50 2022] PUT /identity/v3/projects/a054de6c24364e7abc26f003185755ac/users/436f3c7565f6486ebbaffaf6aec7df23/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:50.866268 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8bd8f-7f56-4c4d-9972-f3e1b5f29ab1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:50.881855 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8bd8f-7f56-4c4d-9972-f3e1b5f29ab1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:50.988840 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce192ba-3017-4b22-9dbb-e4278ee18f28 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:50.988840 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3ce192ba-3017-4b22-9dbb-e4278ee18f28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:51.019984 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867c84a-376a-40be-a7df-c3c7132b12f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:51.020991 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867c84a-376a-40be-a7df-c3c7132b12f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:51.021419 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867c84a-376a-40be-a7df-c3c7132b12f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:51.021894 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867c84a-376a-40be-a7df-c3c7132b12f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:51.022880 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867c84a-376a-40be-a7df-c3c7132b12f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:51.024737 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867c84a-376a-40be-a7df-c3c7132b12f7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:51.050292 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 919/1856] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:51.054208 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867c84a-376a-40be-a7df-c3c7132b12f7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:51.079482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867c84a-376a-40be-a7df-c3c7132b12f7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:51.086490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69cc8c-a0e0-4776-84f1-3ed7bf21be29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:51.090416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69cc8c-a0e0-4776-84f1-3ed7bf21be29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:51.090712 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69cc8c-a0e0-4776-84f1-3ed7bf21be29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:51.091001 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69cc8c-a0e0-4776-84f1-3ed7bf21be29 admin admin] PATH_INFO: `/v3/projects/a054de6c24364e7abc26f003185755ac/users/436f3c7565f6486ebbaffaf6aec7df23/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:51.100862 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba69cc8c-a0e0-4776-84f1-3ed7bf21be29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a054de6c24364e7abc26f003185755ac, user_id=436f3c7565f6486ebbaffaf6aec7df23, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:51.113279 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba69cc8c-a0e0-4776-84f1-3ed7bf21be29 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:51.157088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 920/1857] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:51 2022] PUT /identity/v3/projects/a054de6c24364e7abc26f003185755ac/users/436f3c7565f6486ebbaffaf6aec7df23/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:51.166390 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d535b2-117a-48d1-918a-f2a80bd4ac06 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:51.166390 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d535b2-117a-48d1-918a-f2a80bd4ac06 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:51.166390 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d535b2-117a-48d1-918a-f2a80bd4ac06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:51.219158 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce192ba-3017-4b22-9dbb-e4278ee18f28 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:51.247492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-47d535b2-117a-48d1-918a-f2a80bd4ac06 None None] MFA Rules not processed for user `436f3c7565f6486ebbaffaf6aec7df23`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:51.254885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-47d535b2-117a-48d1-918a-f2a80bd4ac06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:51.323195 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:51.340337 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54fface-dd8a-486d-928a-dbe344b461c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:51.341523 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54fface-dd8a-486d-928a-dbe344b461c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:51.342958 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54fface-dd8a-486d-928a-dbe344b461c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:51.343368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54fface-dd8a-486d-928a-dbe344b461c8 admin admin] PATH_INFO: `/v3/users/f414b8819c2f429e9958bd989ba68123` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:51.345097 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54fface-dd8a-486d-928a-dbe344b461c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f414b8819c2f429e9958bd989ba68123)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:51.345392 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:51.347854 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54fface-dd8a-486d-928a-dbe344b461c8 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:51.399978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 921/1858] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:51.411056 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d535b2-117a-48d1-918a-f2a80bd4ac06 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:51.412995 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-47d535b2-117a-48d1-918a-f2a80bd4ac06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:51.424241 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c54fface-dd8a-486d-928a-dbe344b461c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f414b8819c2f429e9958bd989ba68123'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:51.441328 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c54fface-dd8a-486d-928a-dbe344b461c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f414b8819c2f429e9958bd989ba68123'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:51.461808 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c54fface-dd8a-486d-928a-dbe344b461c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f414b8819c2f429e9958bd989ba68123'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:51.464199 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c54fface-dd8a-486d-928a-dbe344b461c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f414b8819c2f429e9958bd989ba68123'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:51.475321 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c54fface-dd8a-486d-928a-dbe344b461c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f414b8819c2f429e9958bd989ba68123'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:51.508388 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d535b2-117a-48d1-918a-f2a80bd4ac06 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:51.521449 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccc45b-e780-421b-b1eb-8f800ae7c59e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:51.523660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccc45b-e780-421b-b1eb-8f800ae7c59e service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:51.524063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccc45b-e780-421b-b1eb-8f800ae7c59e service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:51.524377 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccc45b-e780-421b-b1eb-8f800ae7c59e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:51.532116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 938/1859] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:21:50 2022] DELETE /identity/v3/users/f414b8819c2f429e9958bd989ba68123 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:21:51.535139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-72ccc45b-e780-421b-b1eb-8f800ae7c59e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:51.537492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595597db-c53f-4f52-8b0b-3f3432e91a7c None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:51.537890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595597db-c53f-4f52-8b0b-3f3432e91a7c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:51.538269 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595597db-c53f-4f52-8b0b-3f3432e91a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:51.569992 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ccc45b-e780-421b-b1eb-8f800ae7c59e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:51.571485 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ccc45b-e780-421b-b1eb-8f800ae7c59e service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:51.589268 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-595597db-c53f-4f52-8b0b-3f3432e91a7c None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:51.594534 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-595597db-c53f-4f52-8b0b-3f3432e91a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:51.603272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 922/1860] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:51 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:51.750106 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 939/1861] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:51.784268 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccc45b-e780-421b-b1eb-8f800ae7c59e service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:51.784268 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-72ccc45b-e780-421b-b1eb-8f800ae7c59e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:51.789958 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595597db-c53f-4f52-8b0b-3f3432e91a7c None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:51.791653 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-595597db-c53f-4f52-8b0b-3f3432e91a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:51.825703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccc45b-e780-421b-b1eb-8f800ae7c59e service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:51.956645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7955574-bd72-4467-8253-385bacc29dc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:51.957592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7955574-bd72-4467-8253-385bacc29dc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:51.957709 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595597db-c53f-4f52-8b0b-3f3432e91a7c None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:51.958170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7955574-bd72-4467-8253-385bacc29dc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:51.958636 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7955574-bd72-4467-8253-385bacc29dc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:51.962322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f7955574-bd72-4467-8253-385bacc29dc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:51.969261 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e90660-14ea-42d8-8aa5-a4cdac99fafb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:51.970862 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e90660-14ea-42d8-8aa5-a4cdac99fafb service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:51.971416 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e90660-14ea-42d8-8aa5-a4cdac99fafb service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:51.972405 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e90660-14ea-42d8-8aa5-a4cdac99fafb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:52.009165 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e90660-14ea-42d8-8aa5-a4cdac99fafb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:52.011097 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e90660-14ea-42d8-8aa5-a4cdac99fafb service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:52.034094 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 940/1862] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:21:52.049386 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063756c-254a-40d6-9e41-a82699c334ba None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:52.050373 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063756c-254a-40d6-9e41-a82699c334ba None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:52.051003 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063756c-254a-40d6-9e41-a82699c334ba None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:52.053368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 941/1863] 10.222.0.81 () {54 vars in 921 bytes} [Tue Jun 7 14:21:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:52.076840 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a3677-ebc0-4a04-889d-837a3ad6fcec None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:52.078045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a3677-ebc0-4a04-889d-837a3ad6fcec None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:52.078611 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a3677-ebc0-4a04-889d-837a3ad6fcec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:52.093886 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7955574-bd72-4467-8253-385bacc29dc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:52.099408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7955574-bd72-4467-8253-385bacc29dc5 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:52.112788 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 923/1864] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:52.116828 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-152a3677-ebc0-4a04-889d-837a3ad6fcec None None] MFA Rules not processed for user `2ae726fbc9cf40e68c7587552a118a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:52.119013 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-152a3677-ebc0-4a04-889d-837a3ad6fcec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:52.131640 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:52.299764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 942/1865] 10.222.0.81 () {58 vars in 1018 bytes} [Tue Jun 7 14:21:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:52.318451 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7955574-bd72-4467-8253-385bacc29dc5 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:52.319936 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f7955574-bd72-4467-8253-385bacc29dc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:52.355204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7955574-bd72-4467-8253-385bacc29dc5 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:52.403611 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61196873-23b6-453a-b40f-e99a327e32fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:52.406331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61196873-23b6-453a-b40f-e99a327e32fb service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:52.406633 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61196873-23b6-453a-b40f-e99a327e32fb service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:52.406919 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61196873-23b6-453a-b40f-e99a327e32fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:52.424996 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a3677-ebc0-4a04-889d-837a3ad6fcec None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:52.426816 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61196873-23b6-453a-b40f-e99a327e32fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:52.427778 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-152a3677-ebc0-4a04-889d-837a3ad6fcec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:52.428483 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61196873-23b6-453a-b40f-e99a327e32fb service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:52.442275 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a3677-ebc0-4a04-889d-837a3ad6fcec None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:52.452723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 924/1866] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:21:52.459102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c677d-3343-4e72-88c8-2d0d225bd29e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:52.460225 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c677d-3343-4e72-88c8-2d0d225bd29e service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:52.460494 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c677d-3343-4e72-88c8-2d0d225bd29e service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:52.460779 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c677d-3343-4e72-88c8-2d0d225bd29e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:52.472105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-370c677d-3343-4e72-88c8-2d0d225bd29e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:52.566853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c677d-3343-4e72-88c8-2d0d225bd29e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:52.568299 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c677d-3343-4e72-88c8-2d0d225bd29e service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:52.584296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61196873-23b6-453a-b40f-e99a327e32fb service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:52.586417 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-61196873-23b6-453a-b40f-e99a327e32fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:52.597269 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 943/1867] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:52 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:21:52.627993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61196873-23b6-453a-b40f-e99a327e32fb service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:52.637230 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a91de0-8715-4209-93ed-52750b346b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:52.638253 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a91de0-8715-4209-93ed-52750b346b7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:52.640568 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a91de0-8715-4209-93ed-52750b346b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:52.640979 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a91de0-8715-4209-93ed-52750b346b7a admin admin] PATH_INFO: `/v3/projects/1d7ba0d0d1e24ec488a585d52dc77a3c` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:52.643284 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a91de0-8715-4209-93ed-52750b346b7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d7ba0d0d1e24ec488a585d52dc77a3c)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:52.645455 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a91de0-8715-4209-93ed-52750b346b7a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:52.674811 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:52.709893 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-83a91de0-8715-4209-93ed-52750b346b7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d7ba0d0d1e24ec488a585d52dc77a3c', 'request_id': 'req-83a91de0-8715-4209-93ed-52750b346b7a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:52.718449 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-83a91de0-8715-4209-93ed-52750b346b7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d7ba0d0d1e24ec488a585d52dc77a3c', 'request_id': 'req-83a91de0-8715-4209-93ed-52750b346b7a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:52.724729 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-83a91de0-8715-4209-93ed-52750b346b7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d7ba0d0d1e24ec488a585d52dc77a3c', 'request_id': 'req-83a91de0-8715-4209-93ed-52750b346b7a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:52.736567 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-83a91de0-8715-4209-93ed-52750b346b7a admin admin] The token cache is being invalidate because project 1d7ba0d0d1e24ec488a585d52dc77a3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:21:52.736859 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-83a91de0-8715-4209-93ed-52750b346b7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:52.737517 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-83a91de0-8715-4209-93ed-52750b346b7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:21:52.739076 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 925/1868] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:21:52 2022] DELETE /identity/v3/projects/1d7ba0d0d1e24ec488a585d52dc77a3c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:21:52.744304 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc4b08-b006-4986-a3d3-c1db4137cb0f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:52.744700 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc4b08-b006-4986-a3d3-c1db4137cb0f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:52.745049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc4b08-b006-4986-a3d3-c1db4137cb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:52.764343 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-bcbc4b08-b006-4986-a3d3-c1db4137cb0f None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:52.765769 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-bcbc4b08-b006-4986-a3d3-c1db4137cb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:52.838411 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 944/1869] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:21:52.852329 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6adc3e-b3fa-435a-86ec-7f9577b35bc0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:52.852651 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6adc3e-b3fa-435a-86ec-7f9577b35bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:52.852920 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6adc3e-b3fa-435a-86ec-7f9577b35bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:52.877788 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5e6adc3e-b3fa-435a-86ec-7f9577b35bc0 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:52.879655 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5e6adc3e-b3fa-435a-86ec-7f9577b35bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:52.890727 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 926/1870] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:52.895329 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc4b08-b006-4986-a3d3-c1db4137cb0f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:52.912567 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc4b08-b006-4986-a3d3-c1db4137cb0f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:52.917960 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ec25c-07fb-4f18-b075-0fd25cf2dcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:52.918751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ec25c-07fb-4f18-b075-0fd25cf2dcb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:52.919126 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ec25c-07fb-4f18-b075-0fd25cf2dcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:52.919429 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ec25c-07fb-4f18-b075-0fd25cf2dcb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:52.920745 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ec25c-07fb-4f18-b075-0fd25cf2dcb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:52.921193 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ec25c-07fb-4f18-b075-0fd25cf2dcb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:52.922694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ec25c-07fb-4f18-b075-0fd25cf2dcb5 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:52.935569 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 945/1871] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:52.940763 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6adc3e-b3fa-435a-86ec-7f9577b35bc0 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:52.950666 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6adc3e-b3fa-435a-86ec-7f9577b35bc0 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:52.955562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ac78e-0e9d-4fae-9690-28aafc8011ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:52.956249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ac78e-0e9d-4fae-9690-28aafc8011ec admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:52.956534 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ac78e-0e9d-4fae-9690-28aafc8011ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:52.956839 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ac78e-0e9d-4fae-9690-28aafc8011ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:52.957541 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ac78e-0e9d-4fae-9690-28aafc8011ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:52.959522 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ac78e-0e9d-4fae-9690-28aafc8011ec admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:52.991629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 927/1872] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:21:52 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:52.992935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ec25c-07fb-4f18-b075-0fd25cf2dcb5 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:53.004337 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ec25c-07fb-4f18-b075-0fd25cf2dcb5 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:53.031472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:53.078411 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876fd9b-d8d0-4cab-bd11-9049ee523d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:53.079177 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876fd9b-d8d0-4cab-bd11-9049ee523d93 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:53.079492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876fd9b-d8d0-4cab-bd11-9049ee523d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:53.079915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876fd9b-d8d0-4cab-bd11-9049ee523d93 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:53.080644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876fd9b-d8d0-4cab-bd11-9049ee523d93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:53.082606 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876fd9b-d8d0-4cab-bd11-9049ee523d93 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:53.105644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 946/1873] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:21:52 2022] POST /identity/v3/users => generated 284 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:21:53.106913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ac78e-0e9d-4fae-9690-28aafc8011ec admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:53.118492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ac78e-0e9d-4fae-9690-28aafc8011ec admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:53.124123 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3b2d5-d547-42d8-8b6a-ebb97fc1f269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:53.124817 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3b2d5-d547-42d8-8b6a-ebb97fc1f269 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:53.125083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3b2d5-d547-42d8-8b6a-ebb97fc1f269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:53.125361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3b2d5-d547-42d8-8b6a-ebb97fc1f269 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:53.126127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3b2d5-d547-42d8-8b6a-ebb97fc1f269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:53.127754 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3b2d5-d547-42d8-8b6a-ebb97fc1f269 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:53.138555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 928/1874] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:53.142510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876fd9b-d8d0-4cab-bd11-9049ee523d93 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:53.152484 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876fd9b-d8d0-4cab-bd11-9049ee523d93 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:53.157724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af4df1-9e26-41cb-b6b7-70db6196f3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:53.158395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af4df1-9e26-41cb-b6b7-70db6196f3f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:53.158683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af4df1-9e26-41cb-b6b7-70db6196f3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:53.158964 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af4df1-9e26-41cb-b6b7-70db6196f3f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:53.159734 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af4df1-9e26-41cb-b6b7-70db6196f3f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:53.161347 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af4df1-9e26-41cb-b6b7-70db6196f3f3 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:53.172579 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 947/1875] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:53.176754 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3b2d5-d547-42d8-8b6a-ebb97fc1f269 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:53.186737 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3b2d5-d547-42d8-8b6a-ebb97fc1f269 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:53.191310 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c2cf5-4c07-4496-876e-14da79b6bd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:53.192211 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c2cf5-4c07-4496-876e-14da79b6bd27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:53.192501 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c2cf5-4c07-4496-876e-14da79b6bd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:53.192779 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c2cf5-4c07-4496-876e-14da79b6bd27 admin admin] PATH_INFO: `/v3/projects/63a4ece4819f4abc8e66fc7c1e209154/users/5b287783bb2c4380b9a122da75a25da8/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:53.207422 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c2cf5-4c07-4496-876e-14da79b6bd27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63a4ece4819f4abc8e66fc7c1e209154, user_id=5b287783bb2c4380b9a122da75a25da8, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:53.210080 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c2cf5-4c07-4496-876e-14da79b6bd27 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:53.223748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 929/1876] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:21:53 2022] PUT /identity/v3/projects/63a4ece4819f4abc8e66fc7c1e209154/users/5b287783bb2c4380b9a122da75a25da8/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:53.234910 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6fbc9-3fa9-4cb9-b653-98e4a8eb383f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:53.235199 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6fbc9-3fa9-4cb9-b653-98e4a8eb383f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:53.235511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6fbc9-3fa9-4cb9-b653-98e4a8eb383f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:53.261447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-2ca6fbc9-3fa9-4cb9-b653-98e4a8eb383f None None] MFA Rules not processed for user `5b287783bb2c4380b9a122da75a25da8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:53.262583 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2ca6fbc9-3fa9-4cb9-b653-98e4a8eb383f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:53.332308 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 948/1877] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:21:53 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:21:53.343572 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c2cf5-4c07-4496-876e-14da79b6bd27 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:53.345323 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-445c2cf5-4c07-4496-876e-14da79b6bd27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:53.406006 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c2cf5-4c07-4496-876e-14da79b6bd27 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:53.410620 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed785cc-f081-4a75-9955-ec13aed5a5e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:53.411482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed785cc-f081-4a75-9955-ec13aed5a5e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:53.411765 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed785cc-f081-4a75-9955-ec13aed5a5e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:53.412033 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed785cc-f081-4a75-9955-ec13aed5a5e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:53.422452 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed785cc-f081-4a75-9955-ec13aed5a5e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:53.423852 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed785cc-f081-4a75-9955-ec13aed5a5e3 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:53.433187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 930/1878] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:21:53 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:01.169993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf8b96-4158-4283-8edb-621fc7b0e4bc None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:01.171183 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf8b96-4158-4283-8edb-621fc7b0e4bc None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:01.172238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf8b96-4158-4283-8edb-621fc7b0e4bc None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:01.173907 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 949/1879] 10.222.0.81 () {54 vars in 921 bytes} [Tue Jun 7 14:22:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:22:01.192400 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919fb62-d6e5-4a9e-b899-761bafcd44b9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:01.192861 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919fb62-d6e5-4a9e-b899-761bafcd44b9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:01.193188 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919fb62-d6e5-4a9e-b899-761bafcd44b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:01.239014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1919fb62-d6e5-4a9e-b899-761bafcd44b9 None None] MFA Rules not processed for user `2ae726fbc9cf40e68c7587552a118a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:01.241009 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1919fb62-d6e5-4a9e-b899-761bafcd44b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:01.335340 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf8b96-4158-4283-8edb-621fc7b0e4bc None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:01.340051 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f4cf8b96-4158-4283-8edb-621fc7b0e4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:01.372826 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:01.421718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 931/1880] 10.222.0.81 () {58 vars in 1018 bytes} [Tue Jun 7 14:22:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:01.431662 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf8b96-4158-4283-8edb-621fc7b0e4bc None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:01.444376 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919fb62-d6e5-4a9e-b899-761bafcd44b9 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:01.446005 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1919fb62-d6e5-4a9e-b899-761bafcd44b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:01.461411 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263eeb7-af1d-4e0a-8e2b-5873ff0fbc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:01.464088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263eeb7-af1d-4e0a-8e2b-5873ff0fbc19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:01.464438 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263eeb7-af1d-4e0a-8e2b-5873ff0fbc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:01.464759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263eeb7-af1d-4e0a-8e2b-5873ff0fbc19 admin admin] PATH_INFO: `/v3/users/7f5966e6a1134057b77d5b1811582d4d` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:01.469450 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4263eeb7-af1d-4e0a-8e2b-5873ff0fbc19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f5966e6a1134057b77d5b1811582d4d)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:01.476157 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4263eeb7-af1d-4e0a-8e2b-5873ff0fbc19 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:01.478721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919fb62-d6e5-4a9e-b899-761bafcd44b9 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:01.492823 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8dfd05-427c-4351-b701-fbc71fcb22dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:01.495416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8dfd05-427c-4351-b701-fbc71fcb22dd service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:01.495675 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8dfd05-427c-4351-b701-fbc71fcb22dd service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:01.495965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8dfd05-427c-4351-b701-fbc71fcb22dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:01.512463 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8dfd05-427c-4351-b701-fbc71fcb22dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:01.514093 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8dfd05-427c-4351-b701-fbc71fcb22dd service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:01.542030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 932/1881] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:22:01.565698 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:01.576815 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4263eeb7-af1d-4e0a-8e2b-5873ff0fbc19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f5966e6a1134057b77d5b1811582d4d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:01.594009 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4263eeb7-af1d-4e0a-8e2b-5873ff0fbc19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f5966e6a1134057b77d5b1811582d4d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:01.595197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4263eeb7-af1d-4e0a-8e2b-5873ff0fbc19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f5966e6a1134057b77d5b1811582d4d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:01.596018 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4263eeb7-af1d-4e0a-8e2b-5873ff0fbc19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f5966e6a1134057b77d5b1811582d4d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:01.618655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4263eeb7-af1d-4e0a-8e2b-5873ff0fbc19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f5966e6a1134057b77d5b1811582d4d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:01.648836 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 950/1882] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:01 2022] DELETE /identity/v3/users/7f5966e6a1134057b77d5b1811582d4d => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:01.651091 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5864cf-a839-46d0-9eea-a6f663895413 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:01.651496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5864cf-a839-46d0-9eea-a6f663895413 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:01.651844 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5864cf-a839-46d0-9eea-a6f663895413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:01.672183 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ab5864cf-a839-46d0-9eea-a6f663895413 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:01.674008 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ab5864cf-a839-46d0-9eea-a6f663895413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:01.749043 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 933/1883] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:01.759487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263eeb7-af1d-4e0a-8e2b-5873ff0fbc19 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:01.761162 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4263eeb7-af1d-4e0a-8e2b-5873ff0fbc19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:01.837947 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263eeb7-af1d-4e0a-8e2b-5873ff0fbc19 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:01.841788 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111f6a9-6a08-4a3b-a467-cbbcca50c200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:01.843141 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111f6a9-6a08-4a3b-a467-cbbcca50c200 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:01.843628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111f6a9-6a08-4a3b-a467-cbbcca50c200 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:01.844092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111f6a9-6a08-4a3b-a467-cbbcca50c200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:01.854793 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111f6a9-6a08-4a3b-a467-cbbcca50c200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:01.856925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111f6a9-6a08-4a3b-a467-cbbcca50c200 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:01.877480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 951/1884] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:02.006234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5864cf-a839-46d0-9eea-a6f663895413 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:02.008738 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ab5864cf-a839-46d0-9eea-a6f663895413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:02.024822 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5864cf-a839-46d0-9eea-a6f663895413 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:02.038305 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73532bf-09c7-4c01-b6f5-73ce4b01f036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:02.040333 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73532bf-09c7-4c01-b6f5-73ce4b01f036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:02.040839 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73532bf-09c7-4c01-b6f5-73ce4b01f036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:02.041733 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73532bf-09c7-4c01-b6f5-73ce4b01f036 admin admin] PATH_INFO: `/v3/users/b6055476a66646e9b493e70a62931e71` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:02.044557 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73532bf-09c7-4c01-b6f5-73ce4b01f036 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6055476a66646e9b493e70a62931e71)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:02.049795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73532bf-09c7-4c01-b6f5-73ce4b01f036 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:02.135478 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d73532bf-09c7-4c01-b6f5-73ce4b01f036 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6055476a66646e9b493e70a62931e71'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:02.136270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d73532bf-09c7-4c01-b6f5-73ce4b01f036 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6055476a66646e9b493e70a62931e71'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:02.137049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d73532bf-09c7-4c01-b6f5-73ce4b01f036 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6055476a66646e9b493e70a62931e71'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:02.161416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d73532bf-09c7-4c01-b6f5-73ce4b01f036 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6055476a66646e9b493e70a62931e71'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:02.184288 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d73532bf-09c7-4c01-b6f5-73ce4b01f036 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6055476a66646e9b493e70a62931e71'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:02.210733 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 934/1885] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:02 2022] DELETE /identity/v3/users/b6055476a66646e9b493e70a62931e71 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:02.215696 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed510329-7fce-4f58-9e28-98b180d2d470 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:02.216650 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed510329-7fce-4f58-9e28-98b180d2d470 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:02.216944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed510329-7fce-4f58-9e28-98b180d2d470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:02.238252 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-ed510329-7fce-4f58-9e28-98b180d2d470 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:02.239636 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ed510329-7fce-4f58-9e28-98b180d2d470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:02.296496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73532bf-09c7-4c01-b6f5-73ce4b01f036 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:02.298248 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d73532bf-09c7-4c01-b6f5-73ce4b01f036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:02.316730 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 952/1886] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:02.344476 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed510329-7fce-4f58-9e28-98b180d2d470 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:02.346099 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ed510329-7fce-4f58-9e28-98b180d2d470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:02.381940 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:02.414769 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73532bf-09c7-4c01-b6f5-73ce4b01f036 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:02.421095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7eefcb-7954-4f6a-86d5-699b6df7f95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:02.422121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7eefcb-7954-4f6a-86d5-699b6df7f95c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:02.422439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7eefcb-7954-4f6a-86d5-699b6df7f95c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:02.422713 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7eefcb-7954-4f6a-86d5-699b6df7f95c admin admin] PATH_INFO: `/v3/projects/86764c2a9c5b4f4b96ea56618f439e08` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:02.423996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7eefcb-7954-4f6a-86d5-699b6df7f95c admin admin] RBAC: Authorizing `identity:delete_project(project_id=86764c2a9c5b4f4b96ea56618f439e08)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:02.425913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7eefcb-7954-4f6a-86d5-699b6df7f95c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:02.439943 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed510329-7fce-4f58-9e28-98b180d2d470 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:02.444326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368e3c5-218b-42cb-83cc-918e6f52ed15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:02.445226 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368e3c5-218b-42cb-83cc-918e6f52ed15 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:02.445509 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368e3c5-218b-42cb-83cc-918e6f52ed15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:02.446184 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368e3c5-218b-42cb-83cc-918e6f52ed15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:02.461111 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6368e3c5-218b-42cb-83cc-918e6f52ed15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:02.462742 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6368e3c5-218b-42cb-83cc-918e6f52ed15 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:02.480613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 953/1887] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:22:02.491265 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0a7eefcb-7954-4f6a-86d5-699b6df7f95c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86764c2a9c5b4f4b96ea56618f439e08', 'request_id': 'req-0a7eefcb-7954-4f6a-86d5-699b6df7f95c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:02.499971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0a7eefcb-7954-4f6a-86d5-699b6df7f95c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86764c2a9c5b4f4b96ea56618f439e08', 'request_id': 'req-0a7eefcb-7954-4f6a-86d5-699b6df7f95c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:02.506195 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0a7eefcb-7954-4f6a-86d5-699b6df7f95c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86764c2a9c5b4f4b96ea56618f439e08', 'request_id': 'req-0a7eefcb-7954-4f6a-86d5-699b6df7f95c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:02.518075 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0a7eefcb-7954-4f6a-86d5-699b6df7f95c admin admin] The token cache is being invalidate because project 86764c2a9c5b4f4b96ea56618f439e08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:02.518361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0a7eefcb-7954-4f6a-86d5-699b6df7f95c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:02.519065 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0a7eefcb-7954-4f6a-86d5-699b6df7f95c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:02.520959 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 935/1888] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:02 2022] DELETE /identity/v3/projects/86764c2a9c5b4f4b96ea56618f439e08 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:22:02.525427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368e3c5-218b-42cb-83cc-918e6f52ed15 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:02.531306 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:02.539195 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368e3c5-218b-42cb-83cc-918e6f52ed15 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:02.614834 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7943416b-75a9-4827-9502-164f54f465aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:02.615764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943416b-75a9-4827-9502-164f54f465aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:02.616045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943416b-75a9-4827-9502-164f54f465aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:02.616319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943416b-75a9-4827-9502-164f54f465aa admin admin] PATH_INFO: `/v3/users/436f3c7565f6486ebbaffaf6aec7df23` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:02.617577 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7943416b-75a9-4827-9502-164f54f465aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=436f3c7565f6486ebbaffaf6aec7df23)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:02.619538 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7943416b-75a9-4827-9502-164f54f465aa admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:02.671652 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7943416b-75a9-4827-9502-164f54f465aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '436f3c7565f6486ebbaffaf6aec7df23'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:02.687035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7943416b-75a9-4827-9502-164f54f465aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '436f3c7565f6486ebbaffaf6aec7df23'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:02.687898 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7943416b-75a9-4827-9502-164f54f465aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '436f3c7565f6486ebbaffaf6aec7df23'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:02.688600 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7943416b-75a9-4827-9502-164f54f465aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '436f3c7565f6486ebbaffaf6aec7df23'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:02.697254 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7943416b-75a9-4827-9502-164f54f465aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '436f3c7565f6486ebbaffaf6aec7df23'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:02.713783 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 954/1889] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:02 2022] DELETE /identity/v3/users/436f3c7565f6486ebbaffaf6aec7df23 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:02.810805 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7eefcb-7954-4f6a-86d5-699b6df7f95c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:02.812443 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0a7eefcb-7954-4f6a-86d5-699b6df7f95c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:02.830649 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7eefcb-7954-4f6a-86d5-699b6df7f95c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:02.895464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c1154-763a-429e-91c1-25333b6c4c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:02.896613 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c1154-763a-429e-91c1-25333b6c4c5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:02.897067 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c1154-763a-429e-91c1-25333b6c4c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:02.897719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c1154-763a-429e-91c1-25333b6c4c5e admin admin] PATH_INFO: `/v3/projects/cec5883c84d642c9a8fe2818636b7a90` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:02.899081 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c1154-763a-429e-91c1-25333b6c4c5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cec5883c84d642c9a8fe2818636b7a90)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:02.901529 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c1154-763a-429e-91c1-25333b6c4c5e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:02.952703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d09c1154-763a-429e-91c1-25333b6c4c5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cec5883c84d642c9a8fe2818636b7a90', 'request_id': 'req-d09c1154-763a-429e-91c1-25333b6c4c5e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:02.961681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d09c1154-763a-429e-91c1-25333b6c4c5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cec5883c84d642c9a8fe2818636b7a90', 'request_id': 'req-d09c1154-763a-429e-91c1-25333b6c4c5e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:02.968476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d09c1154-763a-429e-91c1-25333b6c4c5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cec5883c84d642c9a8fe2818636b7a90', 'request_id': 'req-d09c1154-763a-429e-91c1-25333b6c4c5e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:02.970391 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:02.985404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d09c1154-763a-429e-91c1-25333b6c4c5e admin admin] The token cache is being invalidate because project cec5883c84d642c9a8fe2818636b7a90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:02.986026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d09c1154-763a-429e-91c1-25333b6c4c5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:02.986945 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d09c1154-763a-429e-91c1-25333b6c4c5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:02.988622 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 936/1890] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:02 2022] DELETE /identity/v3/projects/cec5883c84d642c9a8fe2818636b7a90 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:02.992615 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7943416b-75a9-4827-9502-164f54f465aa admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:02.994023 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7943416b-75a9-4827-9502-164f54f465aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:03.007064 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7943416b-75a9-4827-9502-164f54f465aa admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:03.026334 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c1154-763a-429e-91c1-25333b6c4c5e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:03.028720 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d09c1154-763a-429e-91c1-25333b6c4c5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:03.075325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee771017-acd9-429a-9b53-595294a21c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:03.076207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee771017-acd9-429a-9b53-595294a21c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:03.076491 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee771017-acd9-429a-9b53-595294a21c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:03.076795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee771017-acd9-429a-9b53-595294a21c92 admin admin] PATH_INFO: `/v3/users/1670d8f144b543cba319596c681c92f8` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:03.078021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee771017-acd9-429a-9b53-595294a21c92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1670d8f144b543cba319596c681c92f8)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:03.080533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee771017-acd9-429a-9b53-595294a21c92 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:03.102503 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c1154-763a-429e-91c1-25333b6c4c5e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:03.103429 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:03.109367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31541c-9972-40ea-9e35-5f4375825b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:03.110451 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31541c-9972-40ea-9e35-5f4375825b6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:03.111561 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31541c-9972-40ea-9e35-5f4375825b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:03.111954 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31541c-9972-40ea-9e35-5f4375825b6d admin admin] PATH_INFO: `/v3/projects/a054de6c24364e7abc26f003185755ac` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:03.113302 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af31541c-9972-40ea-9e35-5f4375825b6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a054de6c24364e7abc26f003185755ac)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:03.115842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af31541c-9972-40ea-9e35-5f4375825b6d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:03.137215 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ee771017-acd9-429a-9b53-595294a21c92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1670d8f144b543cba319596c681c92f8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:03.158249 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ee771017-acd9-429a-9b53-595294a21c92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1670d8f144b543cba319596c681c92f8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:03.159114 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ee771017-acd9-429a-9b53-595294a21c92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1670d8f144b543cba319596c681c92f8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:03.159817 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ee771017-acd9-429a-9b53-595294a21c92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1670d8f144b543cba319596c681c92f8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:03.167842 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ee771017-acd9-429a-9b53-595294a21c92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1670d8f144b543cba319596c681c92f8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:03.176147 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-af31541c-9972-40ea-9e35-5f4375825b6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a054de6c24364e7abc26f003185755ac', 'request_id': 'req-af31541c-9972-40ea-9e35-5f4375825b6d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:03.181409 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 955/1891] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:02 2022] DELETE /identity/v3/users/1670d8f144b543cba319596c681c92f8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:03.185574 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-af31541c-9972-40ea-9e35-5f4375825b6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a054de6c24364e7abc26f003185755ac', 'request_id': 'req-af31541c-9972-40ea-9e35-5f4375825b6d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:03.191777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-af31541c-9972-40ea-9e35-5f4375825b6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a054de6c24364e7abc26f003185755ac', 'request_id': 'req-af31541c-9972-40ea-9e35-5f4375825b6d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:03.202730 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-af31541c-9972-40ea-9e35-5f4375825b6d admin admin] The token cache is being invalidate because project a054de6c24364e7abc26f003185755ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:03.203024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-af31541c-9972-40ea-9e35-5f4375825b6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:03.203718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-af31541c-9972-40ea-9e35-5f4375825b6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:03.205295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 937/1892] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:03 2022] DELETE /identity/v3/projects/a054de6c24364e7abc26f003185755ac => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:22:03.210006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f9719-ba0f-4e71-ab40-1308cafd64a5 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:03.210297 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f9719-ba0f-4e71-ab40-1308cafd64a5 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:03.210592 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f9719-ba0f-4e71-ab40-1308cafd64a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:03.230215 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-349f9719-ba0f-4e71-ab40-1308cafd64a5 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:03.231563 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-349f9719-ba0f-4e71-ab40-1308cafd64a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:03.296240 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 956/1893] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:22:03.304104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecb771-9bf5-49d6-960f-8149c3c45005 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:03.304625 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecb771-9bf5-49d6-960f-8149c3c45005 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:03.304726 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecb771-9bf5-49d6-960f-8149c3c45005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:03.328853 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-02ecb771-9bf5-49d6-960f-8149c3c45005 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:03.331993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-02ecb771-9bf5-49d6-960f-8149c3c45005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:03.338751 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 938/1894] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:03.343752 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f9719-ba0f-4e71-ab40-1308cafd64a5 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:03.354480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f9719-ba0f-4e71-ab40-1308cafd64a5 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:03.359446 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63716004-831a-4207-9b75-f156efba5ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:03.360614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63716004-831a-4207-9b75-f156efba5ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:03.360614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63716004-831a-4207-9b75-f156efba5ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:03.360946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63716004-831a-4207-9b75-f156efba5ad9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:03.361697 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63716004-831a-4207-9b75-f156efba5ad9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:22:03.362010 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63716004-831a-4207-9b75-f156efba5ad9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:03.363771 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63716004-831a-4207-9b75-f156efba5ad9 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:03.379162 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 957/1895] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:22:03.380425 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecb771-9bf5-49d6-960f-8149c3c45005 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:03.392716 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecb771-9bf5-49d6-960f-8149c3c45005 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:03.397654 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c235f-39b2-471c-85bd-d729e2626268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:03.398320 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c235f-39b2-471c-85bd-d729e2626268 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:03.398603 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c235f-39b2-471c-85bd-d729e2626268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:03.398883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c235f-39b2-471c-85bd-d729e2626268 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:03.399627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c235f-39b2-471c-85bd-d729e2626268 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:03.401561 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c235f-39b2-471c-85bd-d729e2626268 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:03.443969 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 939/1896] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:03 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:03.445596 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63716004-831a-4207-9b75-f156efba5ad9 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:03.461917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63716004-831a-4207-9b75-f156efba5ad9 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:03.473631 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c235f-39b2-471c-85bd-d729e2626268 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:03.477320 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f02c235f-39b2-471c-85bd-d729e2626268 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:03.543385 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee2553-9dda-4369-95b6-57860ecf1d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:03.544433 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee2553-9dda-4369-95b6-57860ecf1d0d admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:03.544720 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee2553-9dda-4369-95b6-57860ecf1d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:03.544995 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee2553-9dda-4369-95b6-57860ecf1d0d admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:03.545714 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee2553-9dda-4369-95b6-57860ecf1d0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:03.547842 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee2553-9dda-4369-95b6-57860ecf1d0d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:03.570425 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:03.573224 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 958/1897] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:03 2022] POST /identity/v3/users => generated 293 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:03.575024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c235f-39b2-471c-85bd-d729e2626268 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:03.578284 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee2553-9dda-4369-95b6-57860ecf1d0d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:03.583471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd5321-f545-434d-b2ed-a8b25a878878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:03.586286 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd5321-f545-434d-b2ed-a8b25a878878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:03.586983 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd5321-f545-434d-b2ed-a8b25a878878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:03.587930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd5321-f545-434d-b2ed-a8b25a878878 admin admin] PATH_INFO: `/v3/projects/0c54eaea9b7c40a38d4b18f3fff8cc70` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:03.589646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecd5321-f545-434d-b2ed-a8b25a878878 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c54eaea9b7c40a38d4b18f3fff8cc70)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:03.590473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee2553-9dda-4369-95b6-57860ecf1d0d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:03.591932 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecd5321-f545-434d-b2ed-a8b25a878878 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:03.596025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90c971-ff63-4b0f-9767-c6ce463079b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:03.596908 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90c971-ff63-4b0f-9767-c6ce463079b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:03.597194 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90c971-ff63-4b0f-9767-c6ce463079b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:03.597636 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90c971-ff63-4b0f-9767-c6ce463079b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:03.598490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90c971-ff63-4b0f-9767-c6ce463079b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:03.600706 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90c971-ff63-4b0f-9767-c6ce463079b9 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:03.615046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 959/1898] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:03.619547 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90c971-ff63-4b0f-9767-c6ce463079b9 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:03.635586 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90c971-ff63-4b0f-9767-c6ce463079b9 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:03.644949 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5de87-c47b-4c7c-91c3-09c7d02e8d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:03.645699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5de87-c47b-4c7c-91c3-09c7d02e8d6f admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:03.645992 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5de87-c47b-4c7c-91c3-09c7d02e8d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:03.646269 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5de87-c47b-4c7c-91c3-09c7d02e8d6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:03.653514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5ecd5321-f545-434d-b2ed-a8b25a878878 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c54eaea9b7c40a38d4b18f3fff8cc70', 'request_id': 'req-5ecd5321-f545-434d-b2ed-a8b25a878878'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:03.655403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5de87-c47b-4c7c-91c3-09c7d02e8d6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:03.660500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5de87-c47b-4c7c-91c3-09c7d02e8d6f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:03.662872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:03.674318 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5ecd5321-f545-434d-b2ed-a8b25a878878 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c54eaea9b7c40a38d4b18f3fff8cc70', 'request_id': 'req-5ecd5321-f545-434d-b2ed-a8b25a878878'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:03.680731 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 960/1899] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:03.684557 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5de87-c47b-4c7c-91c3-09c7d02e8d6f admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:03.685522 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5ecd5321-f545-434d-b2ed-a8b25a878878 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c54eaea9b7c40a38d4b18f3fff8cc70', 'request_id': 'req-5ecd5321-f545-434d-b2ed-a8b25a878878'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:03.698378 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5ecd5321-f545-434d-b2ed-a8b25a878878 admin admin] The token cache is being invalidate because project 0c54eaea9b7c40a38d4b18f3fff8cc70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:03.698673 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5ecd5321-f545-434d-b2ed-a8b25a878878 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:03.698800 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5de87-c47b-4c7c-91c3-09c7d02e8d6f admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:03.699470 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5ecd5321-f545-434d-b2ed-a8b25a878878 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:03.701250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 940/1900] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:03 2022] DELETE /identity/v3/projects/0c54eaea9b7c40a38d4b18f3fff8cc70 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:22:03.705813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3a127-671b-4268-a5aa-26b785e0c196 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:03.706152 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3a127-671b-4268-a5aa-26b785e0c196 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:03.706433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3a127-671b-4268-a5aa-26b785e0c196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:03.735347 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-31f3a127-671b-4268-a5aa-26b785e0c196 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:03.736861 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-31f3a127-671b-4268-a5aa-26b785e0c196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:03.771480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc950a32-93b9-45df-8696-afaee0c38b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:03.772478 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc950a32-93b9-45df-8696-afaee0c38b7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:03.772830 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc950a32-93b9-45df-8696-afaee0c38b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:03.773176 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc950a32-93b9-45df-8696-afaee0c38b7d admin admin] PATH_INFO: `/v3/projects/5f603a7c310b4941825d444cdd17fe14/users/5c4fc844a1a6466c94d14abfc0cec777/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:03.791301 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc950a32-93b9-45df-8696-afaee0c38b7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f603a7c310b4941825d444cdd17fe14, user_id=5c4fc844a1a6466c94d14abfc0cec777, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:03.794023 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc950a32-93b9-45df-8696-afaee0c38b7d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:03.803654 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 941/1901] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:22:03.811456 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 961/1902] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:03 2022] PUT /identity/v3/projects/5f603a7c310b4941825d444cdd17fe14/users/5c4fc844a1a6466c94d14abfc0cec777/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:22:03.813777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16b6fd-2132-4efc-aabd-b8d222a88f59 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:03.814108 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16b6fd-2132-4efc-aabd-b8d222a88f59 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:03.814392 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16b6fd-2132-4efc-aabd-b8d222a88f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:03.823166 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0bf32b-6367-46ea-ab86-dd2afebeccd0 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:03.823487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0bf32b-6367-46ea-ab86-dd2afebeccd0 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:03.823767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0bf32b-6367-46ea-ab86-dd2afebeccd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:03.835152 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-eb16b6fd-2132-4efc-aabd-b8d222a88f59 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:03.836504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-eb16b6fd-2132-4efc-aabd-b8d222a88f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:03.855767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-2e0bf32b-6367-46ea-ab86-dd2afebeccd0 None None] MFA Rules not processed for user `5c4fc844a1a6466c94d14abfc0cec777`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:03.863680 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2e0bf32b-6367-46ea-ab86-dd2afebeccd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:03.903054 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 942/1903] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:03.908058 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb16b6fd-2132-4efc-aabd-b8d222a88f59 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:03.917325 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb16b6fd-2132-4efc-aabd-b8d222a88f59 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:03.922389 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ffbe0-005f-4741-9d7a-0e78ce468173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:03.923178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ffbe0-005f-4741-9d7a-0e78ce468173 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:03.923508 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ffbe0-005f-4741-9d7a-0e78ce468173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:03.923807 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ffbe0-005f-4741-9d7a-0e78ce468173 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:03.924535 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ffbe0-005f-4741-9d7a-0e78ce468173 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:22:03.924837 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ffbe0-005f-4741-9d7a-0e78ce468173 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:03.926415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ffbe0-005f-4741-9d7a-0e78ce468173 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:03.939539 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 943/1904] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:22:03.943725 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ffbe0-005f-4741-9d7a-0e78ce468173 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:03.948488 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 962/1905] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:03.954496 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0bf32b-6367-46ea-ab86-dd2afebeccd0 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:03.955952 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2e0bf32b-6367-46ea-ab86-dd2afebeccd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:03.956278 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ffbe0-005f-4741-9d7a-0e78ce468173 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:03.961001 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87621acc-f966-40b3-892f-0b8f280bcc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:03.961651 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87621acc-f966-40b3-892f-0b8f280bcc25 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:03.961934 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87621acc-f966-40b3-892f-0b8f280bcc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:03.962212 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87621acc-f966-40b3-892f-0b8f280bcc25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:03.962902 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87621acc-f966-40b3-892f-0b8f280bcc25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:03.964989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87621acc-f966-40b3-892f-0b8f280bcc25 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:03.969605 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0bf32b-6367-46ea-ab86-dd2afebeccd0 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:03.975872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09963c23-103a-4e46-8da2-5b6d2694243d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:03.977909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09963c23-103a-4e46-8da2-5b6d2694243d admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:03.978192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09963c23-103a-4e46-8da2-5b6d2694243d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:03.978471 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09963c23-103a-4e46-8da2-5b6d2694243d admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:03.979187 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09963c23-103a-4e46-8da2-5b6d2694243d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:03.981195 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09963c23-103a-4e46-8da2-5b6d2694243d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:03.981316 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:03.999512 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 944/1906] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:03 2022] POST /identity/v3/projects => generated 357 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:22:04.001684 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87621acc-f966-40b3-892f-0b8f280bcc25 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:04.008665 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 963/1907] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:03 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:22:04.012702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09963c23-103a-4e46-8da2-5b6d2694243d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:04.061163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:04.076790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09963c23-103a-4e46-8da2-5b6d2694243d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:04.080145 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87621acc-f966-40b3-892f-0b8f280bcc25 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:04.082231 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb88f28-12fb-4a14-9343-b5ae5f42ac79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:04.082930 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb88f28-12fb-4a14-9343-b5ae5f42ac79 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:04.083212 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb88f28-12fb-4a14-9343-b5ae5f42ac79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:04.083546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb88f28-12fb-4a14-9343-b5ae5f42ac79 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:04.084236 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb88f28-12fb-4a14-9343-b5ae5f42ac79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:04.084994 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b51473-fbe1-4bc7-b793-e99ab5eb206c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:04.085672 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b51473-fbe1-4bc7-b793-e99ab5eb206c admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:04.085953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b51473-fbe1-4bc7-b793-e99ab5eb206c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:04.086214 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb88f28-12fb-4a14-9343-b5ae5f42ac79 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:04.086318 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b51473-fbe1-4bc7-b793-e99ab5eb206c admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:04.086917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b51473-fbe1-4bc7-b793-e99ab5eb206c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:04.089215 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b51473-fbe1-4bc7-b793-e99ab5eb206c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:04.107031 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 964/1908] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:04 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:22:04.111642 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 945/1909] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:04 2022] POST /identity/v3/users => generated 279 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:04.111907 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb88f28-12fb-4a14-9343-b5ae5f42ac79 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:04.115628 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b51473-fbe1-4bc7-b793-e99ab5eb206c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:04.121602 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb88f28-12fb-4a14-9343-b5ae5f42ac79 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:04.125784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b51473-fbe1-4bc7-b793-e99ab5eb206c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:04.126846 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2326d7-8279-4137-afc9-7fbe23726e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:04.127555 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2326d7-8279-4137-afc9-7fbe23726e28 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:04.127840 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2326d7-8279-4137-afc9-7fbe23726e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:04.128115 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2326d7-8279-4137-afc9-7fbe23726e28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:04.128844 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2326d7-8279-4137-afc9-7fbe23726e28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:04.130486 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2326d7-8279-4137-afc9-7fbe23726e28 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:04.130608 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9240fa9-6dab-4ecd-a199-9e8ac168ca91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:04.131205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9240fa9-6dab-4ecd-a199-9e8ac168ca91 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:04.131518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9240fa9-6dab-4ecd-a199-9e8ac168ca91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:04.131789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9240fa9-6dab-4ecd-a199-9e8ac168ca91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:04.132699 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9240fa9-6dab-4ecd-a199-9e8ac168ca91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:04.134904 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9240fa9-6dab-4ecd-a199-9e8ac168ca91 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:04.141202 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 965/1910] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:22:04.145210 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2326d7-8279-4137-afc9-7fbe23726e28 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:04.145689 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 946/1911] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:22:04.149949 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9240fa9-6dab-4ecd-a199-9e8ac168ca91 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:04.158327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2326d7-8279-4137-afc9-7fbe23726e28 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:04.162013 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9240fa9-6dab-4ecd-a199-9e8ac168ca91 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:04.163033 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a71aef-7a4e-424d-ad5b-6a1693ca0de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:04.164052 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a71aef-7a4e-424d-ad5b-6a1693ca0de2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:04.164400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a71aef-7a4e-424d-ad5b-6a1693ca0de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:04.164621 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a71aef-7a4e-424d-ad5b-6a1693ca0de2 admin admin] PATH_INFO: `/v3/projects/ec5e3d8e5fde4bf9bbdfdc8d5816373f/users/bfa3a3e67a584123b26d5a83fb6c6e2e/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:04.168783 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4ede1-a3fc-434d-82c9-0baeae4c3368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:04.169043 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4ede1-a3fc-434d-82c9-0baeae4c3368 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:04.169146 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4ede1-a3fc-434d-82c9-0baeae4c3368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:04.169425 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4ede1-a3fc-434d-82c9-0baeae4c3368 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:04.170197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4ede1-a3fc-434d-82c9-0baeae4c3368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:04.172462 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4ede1-a3fc-434d-82c9-0baeae4c3368 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:04.181098 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a71aef-7a4e-424d-ad5b-6a1693ca0de2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec5e3d8e5fde4bf9bbdfdc8d5816373f, user_id=bfa3a3e67a584123b26d5a83fb6c6e2e, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:04.183831 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a71aef-7a4e-424d-ad5b-6a1693ca0de2 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:04.185334 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 947/1912] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:04.189940 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4ede1-a3fc-434d-82c9-0baeae4c3368 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:04.198389 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 966/1913] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:04 2022] PUT /identity/v3/projects/ec5e3d8e5fde4bf9bbdfdc8d5816373f/users/bfa3a3e67a584123b26d5a83fb6c6e2e/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:22:04.201465 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4ede1-a3fc-434d-82c9-0baeae4c3368 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:04.202283 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a71aef-7a4e-424d-ad5b-6a1693ca0de2 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:04.263718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daeb75d-1cb2-4eb9-8e32-5d9e216d403c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:04.264838 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeb75d-1cb2-4eb9-8e32-5d9e216d403c admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:04.265127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeb75d-1cb2-4eb9-8e32-5d9e216d403c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:04.265408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeb75d-1cb2-4eb9-8e32-5d9e216d403c admin admin] PATH_INFO: `/v3/projects/51dd501093ad4b5484ec28131c248876/users/e459020eded5417584bb322d72f80884/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:04.269048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a71aef-7a4e-424d-ad5b-6a1693ca0de2 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:04.274510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a19c0-747e-442f-a711-999a81704ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:04.275178 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a19c0-747e-442f-a711-999a81704ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:04.275485 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a19c0-747e-442f-a711-999a81704ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:04.275758 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a19c0-747e-442f-a711-999a81704ec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:04.276481 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a19c0-747e-442f-a711-999a81704ec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:04.278057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a19c0-747e-442f-a711-999a81704ec8 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:04.279681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daeb75d-1cb2-4eb9-8e32-5d9e216d403c admin admin] RBAC: Authorizing `identity:create_grant(project_id=51dd501093ad4b5484ec28131c248876, user_id=e459020eded5417584bb322d72f80884, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:04.282394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daeb75d-1cb2-4eb9-8e32-5d9e216d403c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:04.288843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 967/1914] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:22:04.293220 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a19c0-747e-442f-a711-999a81704ec8 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:04.298811 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 948/1915] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:04 2022] PUT /identity/v3/projects/51dd501093ad4b5484ec28131c248876/users/e459020eded5417584bb322d72f80884/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:04.303859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a19c0-747e-442f-a711-999a81704ec8 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:04.309111 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ddd6a-8548-4ec9-8434-bde2555b3c50 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:04.309111 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ddd6a-8548-4ec9-8434-bde2555b3c50 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:04.309704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ddd6a-8548-4ec9-8434-bde2555b3c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:04.335733 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-fa9ddd6a-8548-4ec9-8434-bde2555b3c50 None None] MFA Rules not processed for user `e459020eded5417584bb322d72f80884`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:04.336942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fa9ddd6a-8548-4ec9-8434-bde2555b3c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:04.362337 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b73ebb-c2cd-4a4f-b490-9417777287dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:04.363235 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b73ebb-c2cd-4a4f-b490-9417777287dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:04.363529 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b73ebb-c2cd-4a4f-b490-9417777287dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:04.363806 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b73ebb-c2cd-4a4f-b490-9417777287dd admin admin] PATH_INFO: `/v3/projects/ec5e3d8e5fde4bf9bbdfdc8d5816373f/users/bfa3a3e67a584123b26d5a83fb6c6e2e/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:04.365735 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b73ebb-c2cd-4a4f-b490-9417777287dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec5e3d8e5fde4bf9bbdfdc8d5816373f, user_id=bfa3a3e67a584123b26d5a83fb6c6e2e, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:04.368590 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b73ebb-c2cd-4a4f-b490-9417777287dd admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:04.371295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:04.380174 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 968/1916] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:04 2022] PUT /identity/v3/projects/ec5e3d8e5fde4bf9bbdfdc8d5816373f/users/bfa3a3e67a584123b26d5a83fb6c6e2e/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:22:04.390210 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb5eed-332b-4a73-ba0d-46189097c155 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:04.390494 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb5eed-332b-4a73-ba0d-46189097c155 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:04.390910 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb5eed-332b-4a73-ba0d-46189097c155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:04.406624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 949/1917] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:04 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:22:04.411338 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ddd6a-8548-4ec9-8434-bde2555b3c50 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:04.417555 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-adeb5eed-332b-4a73-ba0d-46189097c155 None None] MFA Rules not processed for user `bfa3a3e67a584123b26d5a83fb6c6e2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:04.418748 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-adeb5eed-332b-4a73-ba0d-46189097c155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:04.422354 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ddd6a-8548-4ec9-8434-bde2555b3c50 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:04.437612 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:04.486747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee26168-e852-403b-b004-259e5a40ab2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:04.487659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee26168-e852-403b-b004-259e5a40ab2d admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:04.488030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee26168-e852-403b-b004-259e5a40ab2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:04.488314 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee26168-e852-403b-b004-259e5a40ab2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:04.489011 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee26168-e852-403b-b004-259e5a40ab2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:04.490966 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee26168-e852-403b-b004-259e5a40ab2d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:04.505765 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 969/1918] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:22:04.516569 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb5eed-332b-4a73-ba0d-46189097c155 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:04.517982 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-adeb5eed-332b-4a73-ba0d-46189097c155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:04.521064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 950/1919] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:04 2022] POST /identity/v3/projects => generated 357 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:22:04.522143 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee26168-e852-403b-b004-259e5a40ab2d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:04.574995 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb5eed-332b-4a73-ba0d-46189097c155 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:04.584737 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee26168-e852-403b-b004-259e5a40ab2d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:04.590021 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c442ff-60f6-476e-b91f-c10199e7ccd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:04.590698 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c442ff-60f6-476e-b91f-c10199e7ccd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:04.590989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c442ff-60f6-476e-b91f-c10199e7ccd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:04.591306 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c442ff-60f6-476e-b91f-c10199e7ccd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:04.593047 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c442ff-60f6-476e-b91f-c10199e7ccd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:04.595784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c442ff-60f6-476e-b91f-c10199e7ccd2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:04.621002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 951/1920] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:04 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:22:04.622072 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c442ff-60f6-476e-b91f-c10199e7ccd2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:04.630632 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83e415-f8c4-4240-9966-b3ab7365890b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:04.631468 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83e415-f8c4-4240-9966-b3ab7365890b service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:04.631817 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83e415-f8c4-4240-9966-b3ab7365890b service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:04.632065 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83e415-f8c4-4240-9966-b3ab7365890b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:04.632226 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c442ff-60f6-476e-b91f-c10199e7ccd2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:04.637034 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d118a7-25c4-4ebb-8097-211093d0b5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:04.637728 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d118a7-25c4-4ebb-8097-211093d0b5fb admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:04.638018 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d118a7-25c4-4ebb-8097-211093d0b5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:04.638296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d118a7-25c4-4ebb-8097-211093d0b5fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:04.639032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d118a7-25c4-4ebb-8097-211093d0b5fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:04.640732 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d118a7-25c4-4ebb-8097-211093d0b5fb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:04.651239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 952/1921] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:22:04.655959 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d118a7-25c4-4ebb-8097-211093d0b5fb admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:04.665562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d118a7-25c4-4ebb-8097-211093d0b5fb admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:04.670370 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec7a5d-ddd7-460c-ac01-be0ad19d226f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:04.672013 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec7a5d-ddd7-460c-ac01-be0ad19d226f admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:04.672349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec7a5d-ddd7-460c-ac01-be0ad19d226f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:04.672630 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec7a5d-ddd7-460c-ac01-be0ad19d226f admin admin] PATH_INFO: `/v3/projects/044a09a891044f21a58b8979c2bf5a40/users/a8c64cb736e34d25ba0ad1ad42c30364/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:04.688635 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec7a5d-ddd7-460c-ac01-be0ad19d226f admin admin] RBAC: Authorizing `identity:create_grant(project_id=044a09a891044f21a58b8979c2bf5a40, user_id=a8c64cb736e34d25ba0ad1ad42c30364, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:04.691712 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec7a5d-ddd7-460c-ac01-be0ad19d226f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:04.699092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd83e415-f8c4-4240-9966-b3ab7365890b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:04.700570 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd83e415-f8c4-4240-9966-b3ab7365890b service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:04.704628 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 953/1922] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:04 2022] PUT /identity/v3/projects/044a09a891044f21a58b8979c2bf5a40/users/a8c64cb736e34d25ba0ad1ad42c30364/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:22:04.708581 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec7a5d-ddd7-460c-ac01-be0ad19d226f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:04.710438 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 970/1923] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:22:04.723074 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:04.817601 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec7a5d-ddd7-460c-ac01-be0ad19d226f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:04.822738 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e2ec6-c9df-43ba-a683-6a9241183792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:04.823060 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e2ec6-c9df-43ba-a683-6a9241183792 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:04.823375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e2ec6-c9df-43ba-a683-6a9241183792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:04.823781 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e2ec6-c9df-43ba-a683-6a9241183792 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:04.824518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e2ec6-c9df-43ba-a683-6a9241183792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:04.826118 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e2ec6-c9df-43ba-a683-6a9241183792 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:04.844815 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 954/1924] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:22:04.849654 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83e415-f8c4-4240-9966-b3ab7365890b service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:04.862784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83e415-f8c4-4240-9966-b3ab7365890b service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:04.868737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22b682-73aa-4ccc-9e10-28e800959f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:04.869795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22b682-73aa-4ccc-9e10-28e800959f50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:04.870183 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22b682-73aa-4ccc-9e10-28e800959f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:04.870560 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22b682-73aa-4ccc-9e10-28e800959f50 admin admin] PATH_INFO: `/v3/projects/044a09a891044f21a58b8979c2bf5a40/users/a8c64cb736e34d25ba0ad1ad42c30364/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:04.872792 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e22b682-73aa-4ccc-9e10-28e800959f50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=044a09a891044f21a58b8979c2bf5a40, user_id=a8c64cb736e34d25ba0ad1ad42c30364, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:04.879420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e22b682-73aa-4ccc-9e10-28e800959f50 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:04.896246 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 971/1925] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:04 2022] PUT /identity/v3/projects/044a09a891044f21a58b8979c2bf5a40/users/a8c64cb736e34d25ba0ad1ad42c30364/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:22:04.907948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb3f33-249c-45e8-a3a6-d00e3ae375cf None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:04.908238 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb3f33-249c-45e8-a3a6-d00e3ae375cf None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:04.908520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb3f33-249c-45e8-a3a6-d00e3ae375cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:04.942114 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d7cb3f33-249c-45e8-a3a6-d00e3ae375cf None None] MFA Rules not processed for user `a8c64cb736e34d25ba0ad1ad42c30364`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:04.945985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d7cb3f33-249c-45e8-a3a6-d00e3ae375cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:05.046888 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 955/1926] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:04 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:05.060209 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22b682-73aa-4ccc-9e10-28e800959f50 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:05.063331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1e22b682-73aa-4ccc-9e10-28e800959f50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:05.083631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:05.144932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22b682-73aa-4ccc-9e10-28e800959f50 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:05.149332 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9525ce1-0e4b-45aa-87c4-287821cb76ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:05.149992 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9525ce1-0e4b-45aa-87c4-287821cb76ac service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:05.150729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9525ce1-0e4b-45aa-87c4-287821cb76ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:05.151014 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9525ce1-0e4b-45aa-87c4-287821cb76ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:05.170776 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9525ce1-0e4b-45aa-87c4-287821cb76ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:05.172552 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9525ce1-0e4b-45aa-87c4-287821cb76ac service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:05.201587 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 972/1927] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:05 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:05.326359 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb3f33-249c-45e8-a3a6-d00e3ae375cf None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:05.337271 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb3f33-249c-45e8-a3a6-d00e3ae375cf None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:05.342014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147cce5-84b5-46ff-8ee0-8e63476f4e13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:05.343091 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147cce5-84b5-46ff-8ee0-8e63476f4e13 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:05.343389 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147cce5-84b5-46ff-8ee0-8e63476f4e13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:05.343728 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147cce5-84b5-46ff-8ee0-8e63476f4e13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:05.435047 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f147cce5-84b5-46ff-8ee0-8e63476f4e13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:05.439011 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f147cce5-84b5-46ff-8ee0-8e63476f4e13 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:05.451835 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 956/1928] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:05 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:05.610869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9525ce1-0e4b-45aa-87c4-287821cb76ac service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:05.697366 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9525ce1-0e4b-45aa-87c4-287821cb76ac service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:05.703342 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550a54b-895d-45d4-9999-63390c55a2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:05.704345 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550a54b-895d-45d4-9999-63390c55a2b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:05.705002 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550a54b-895d-45d4-9999-63390c55a2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:05.705002 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550a54b-895d-45d4-9999-63390c55a2b9 admin admin] PATH_INFO: `/v3/users/5c4fc844a1a6466c94d14abfc0cec777` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:05.706268 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550a54b-895d-45d4-9999-63390c55a2b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c4fc844a1a6466c94d14abfc0cec777)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:05.708716 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550a54b-895d-45d4-9999-63390c55a2b9 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:05.816294 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7550a54b-895d-45d4-9999-63390c55a2b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c4fc844a1a6466c94d14abfc0cec777'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:05.837126 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7550a54b-895d-45d4-9999-63390c55a2b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c4fc844a1a6466c94d14abfc0cec777'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:05.838124 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7550a54b-895d-45d4-9999-63390c55a2b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c4fc844a1a6466c94d14abfc0cec777'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:05.838748 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7550a54b-895d-45d4-9999-63390c55a2b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c4fc844a1a6466c94d14abfc0cec777'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:05.850987 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7550a54b-895d-45d4-9999-63390c55a2b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c4fc844a1a6466c94d14abfc0cec777'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:05.880616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 973/1929] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:05 2022] DELETE /identity/v3/users/5c4fc844a1a6466c94d14abfc0cec777 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:05.885706 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca65a5b-4f6f-4ec8-bb41-2bb42ed070a5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:05.885908 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca65a5b-4f6f-4ec8-bb41-2bb42ed070a5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:05.886539 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca65a5b-4f6f-4ec8-bb41-2bb42ed070a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:05.935463 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2ca65a5b-4f6f-4ec8-bb41-2bb42ed070a5 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:05.938570 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2ca65a5b-4f6f-4ec8-bb41-2bb42ed070a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:05.983852 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:06.038488 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 957/1930] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:06.055154 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550a54b-895d-45d4-9999-63390c55a2b9 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:06.055154 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7550a54b-895d-45d4-9999-63390c55a2b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:06.117403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:06.174081 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550a54b-895d-45d4-9999-63390c55a2b9 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:06.180325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad7ea9-c371-4673-95d5-394e8381187c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:06.181248 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad7ea9-c371-4673-95d5-394e8381187c service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:06.181604 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad7ea9-c371-4673-95d5-394e8381187c service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:06.181951 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad7ea9-c371-4673-95d5-394e8381187c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:06.196496 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad7ea9-c371-4673-95d5-394e8381187c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:06.197942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad7ea9-c371-4673-95d5-394e8381187c service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:06.232103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 974/1931] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:06.721173 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca65a5b-4f6f-4ec8-bb41-2bb42ed070a5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:06.722820 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2ca65a5b-4f6f-4ec8-bb41-2bb42ed070a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:06.739693 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca65a5b-4f6f-4ec8-bb41-2bb42ed070a5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:06.745040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e37fe-3fcb-46c9-b1f1-df0caefe8f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:06.746017 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e37fe-3fcb-46c9-b1f1-df0caefe8f72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:06.746321 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e37fe-3fcb-46c9-b1f1-df0caefe8f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:06.746685 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e37fe-3fcb-46c9-b1f1-df0caefe8f72 admin admin] PATH_INFO: `/v3/projects/5f603a7c310b4941825d444cdd17fe14` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:06.747886 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e37fe-3fcb-46c9-b1f1-df0caefe8f72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f603a7c310b4941825d444cdd17fe14)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:06.750162 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e37fe-3fcb-46c9-b1f1-df0caefe8f72 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:06.805102 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f48e37fe-3fcb-46c9-b1f1-df0caefe8f72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f603a7c310b4941825d444cdd17fe14', 'request_id': 'req-f48e37fe-3fcb-46c9-b1f1-df0caefe8f72'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:06.813945 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f48e37fe-3fcb-46c9-b1f1-df0caefe8f72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f603a7c310b4941825d444cdd17fe14', 'request_id': 'req-f48e37fe-3fcb-46c9-b1f1-df0caefe8f72'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:06.821315 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f48e37fe-3fcb-46c9-b1f1-df0caefe8f72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f603a7c310b4941825d444cdd17fe14', 'request_id': 'req-f48e37fe-3fcb-46c9-b1f1-df0caefe8f72'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:06.832549 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f48e37fe-3fcb-46c9-b1f1-df0caefe8f72 admin admin] The token cache is being invalidate because project 5f603a7c310b4941825d444cdd17fe14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:06.832976 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f48e37fe-3fcb-46c9-b1f1-df0caefe8f72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:06.833721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f48e37fe-3fcb-46c9-b1f1-df0caefe8f72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:06.838470 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 958/1932] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:06 2022] DELETE /identity/v3/projects/5f603a7c310b4941825d444cdd17fe14 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:06.840275 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad7ea9-c371-4673-95d5-394e8381187c service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:06.855694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad7ea9-c371-4673-95d5-394e8381187c service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:06.930751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af8659-00b9-494a-b0f1-ab1fcd399fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:06.931956 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af8659-00b9-494a-b0f1-ab1fcd399fb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:06.932369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af8659-00b9-494a-b0f1-ab1fcd399fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:06.932771 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af8659-00b9-494a-b0f1-ab1fcd399fb4 admin admin] PATH_INFO: `/v3/users/bfa3a3e67a584123b26d5a83fb6c6e2e` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:06.934244 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97af8659-00b9-494a-b0f1-ab1fcd399fb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfa3a3e67a584123b26d5a83fb6c6e2e)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:06.937267 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97af8659-00b9-494a-b0f1-ab1fcd399fb4 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:06.998693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-97af8659-00b9-494a-b0f1-ab1fcd399fb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfa3a3e67a584123b26d5a83fb6c6e2e'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:07.011772 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-97af8659-00b9-494a-b0f1-ab1fcd399fb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfa3a3e67a584123b26d5a83fb6c6e2e'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:07.012607 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-97af8659-00b9-494a-b0f1-ab1fcd399fb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfa3a3e67a584123b26d5a83fb6c6e2e'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:07.013205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-97af8659-00b9-494a-b0f1-ab1fcd399fb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfa3a3e67a584123b26d5a83fb6c6e2e'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:07.022346 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-97af8659-00b9-494a-b0f1-ab1fcd399fb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfa3a3e67a584123b26d5a83fb6c6e2e'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:07.039405 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 975/1933] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:06 2022] DELETE /identity/v3/users/bfa3a3e67a584123b26d5a83fb6c6e2e => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:07.477079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e37fe-3fcb-46c9-b1f1-df0caefe8f72 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:07.484029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f48e37fe-3fcb-46c9-b1f1-df0caefe8f72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:07.513014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e37fe-3fcb-46c9-b1f1-df0caefe8f72 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:07.595541 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fd9e9-96b1-4e0a-b746-5109cdd892cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:07.596478 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fd9e9-96b1-4e0a-b746-5109cdd892cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:07.596747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fd9e9-96b1-4e0a-b746-5109cdd892cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:07.597158 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fd9e9-96b1-4e0a-b746-5109cdd892cc admin admin] PATH_INFO: `/v3/projects/ec5e3d8e5fde4bf9bbdfdc8d5816373f` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:07.598353 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811fd9e9-96b1-4e0a-b746-5109cdd892cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec5e3d8e5fde4bf9bbdfdc8d5816373f)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:07.600774 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811fd9e9-96b1-4e0a-b746-5109cdd892cc admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:07.662229 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-811fd9e9-96b1-4e0a-b746-5109cdd892cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec5e3d8e5fde4bf9bbdfdc8d5816373f', 'request_id': 'req-811fd9e9-96b1-4e0a-b746-5109cdd892cc'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:07.671756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-811fd9e9-96b1-4e0a-b746-5109cdd892cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec5e3d8e5fde4bf9bbdfdc8d5816373f', 'request_id': 'req-811fd9e9-96b1-4e0a-b746-5109cdd892cc'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:07.674468 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:07.680850 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-811fd9e9-96b1-4e0a-b746-5109cdd892cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec5e3d8e5fde4bf9bbdfdc8d5816373f', 'request_id': 'req-811fd9e9-96b1-4e0a-b746-5109cdd892cc'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:07.697630 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-811fd9e9-96b1-4e0a-b746-5109cdd892cc admin admin] The token cache is being invalidate because project ec5e3d8e5fde4bf9bbdfdc8d5816373f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:07.698032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-811fd9e9-96b1-4e0a-b746-5109cdd892cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:07.699146 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-811fd9e9-96b1-4e0a-b746-5109cdd892cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:07.701205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 959/1934] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:07 2022] DELETE /identity/v3/projects/ec5e3d8e5fde4bf9bbdfdc8d5816373f => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:07.706146 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f1128-b5c5-4a11-99b3-5229e1962540 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:07.706414 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f1128-b5c5-4a11-99b3-5229e1962540 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:07.706706 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f1128-b5c5-4a11-99b3-5229e1962540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:07.741493 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-0e4f1128-b5c5-4a11-99b3-5229e1962540 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:07.742732 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0e4f1128-b5c5-4a11-99b3-5229e1962540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:07.844330 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:07.873040 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 976/1935] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:07.881280 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57218b-8303-443a-b9ee-6d027b643a71 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:07.882116 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57218b-8303-443a-b9ee-6d027b643a71 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:07.882529 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57218b-8303-443a-b9ee-6d027b643a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:07.916386 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8e57218b-8303-443a-b9ee-6d027b643a71 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:07.917724 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8e57218b-8303-443a-b9ee-6d027b643a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:07.931264 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 960/1936] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:22:07.933213 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f1128-b5c5-4a11-99b3-5229e1962540 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:07.949121 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f1128-b5c5-4a11-99b3-5229e1962540 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:07.954819 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f9ef6b-a30a-4e84-854a-4514124f558e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:07.955799 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9ef6b-a30a-4e84-854a-4514124f558e admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:07.956083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9ef6b-a30a-4e84-854a-4514124f558e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:07.956588 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9ef6b-a30a-4e84-854a-4514124f558e admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:07.957351 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f9ef6b-a30a-4e84-854a-4514124f558e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:22:07.957655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f9ef6b-a30a-4e84-854a-4514124f558e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:07.959505 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f9ef6b-a30a-4e84-854a-4514124f558e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:07.974548 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 977/1937] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:22:07.980973 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57218b-8303-443a-b9ee-6d027b643a71 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:08.001909 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57218b-8303-443a-b9ee-6d027b643a71 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:08.008476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c22c548-1f2e-4553-858f-c025b2b6dcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:08.009201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22c548-1f2e-4553-858f-c025b2b6dcbb admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:08.009464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22c548-1f2e-4553-858f-c025b2b6dcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:08.009739 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22c548-1f2e-4553-858f-c025b2b6dcbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:08.010432 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c22c548-1f2e-4553-858f-c025b2b6dcbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:08.012370 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c22c548-1f2e-4553-858f-c025b2b6dcbb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:08.043236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 961/1938] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:07 2022] POST /identity/v3/projects => generated 369 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:08.046659 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f9ef6b-a30a-4e84-854a-4514124f558e admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:08.065078 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f9ef6b-a30a-4e84-854a-4514124f558e admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:08.141017 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0239b7-2bab-4046-a7a4-4a43c3fc77b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:08.141722 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0239b7-2bab-4046-a7a4-4a43c3fc77b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:08.142008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0239b7-2bab-4046-a7a4-4a43c3fc77b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:08.142305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0239b7-2bab-4046-a7a4-4a43c3fc77b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:08.143004 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0239b7-2bab-4046-a7a4-4a43c3fc77b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:08.145034 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0239b7-2bab-4046-a7a4-4a43c3fc77b3 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:08.169470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 978/1939] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:08 2022] POST /identity/v3/users => generated 285 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:08.173523 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c22c548-1f2e-4553-858f-c025b2b6dcbb admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:08.183713 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c22c548-1f2e-4553-858f-c025b2b6dcbb admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:08.189220 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce3993-ba6d-42ad-a072-63b77a09dc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:08.189873 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce3993-ba6d-42ad-a072-63b77a09dc0a admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:08.190150 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce3993-ba6d-42ad-a072-63b77a09dc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:08.190428 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce3993-ba6d-42ad-a072-63b77a09dc0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:08.191204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce3993-ba6d-42ad-a072-63b77a09dc0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:08.192879 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce3993-ba6d-42ad-a072-63b77a09dc0a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:08.205193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 962/1940] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:08.209146 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0239b7-2bab-4046-a7a4-4a43c3fc77b3 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:08.219892 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0239b7-2bab-4046-a7a4-4a43c3fc77b3 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:08.224899 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdead4-0105-4dda-b92c-d190c8b70184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:08.225864 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdead4-0105-4dda-b92c-d190c8b70184 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:08.226146 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdead4-0105-4dda-b92c-d190c8b70184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:08.226459 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdead4-0105-4dda-b92c-d190c8b70184 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:08.227238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cdead4-0105-4dda-b92c-d190c8b70184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:08.228954 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cdead4-0105-4dda-b92c-d190c8b70184 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:08.241901 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 979/1941] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:22:08.246261 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce3993-ba6d-42ad-a072-63b77a09dc0a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:08.259438 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce3993-ba6d-42ad-a072-63b77a09dc0a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:08.265574 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755589d8-5728-4037-9a62-09497be781d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:08.266674 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755589d8-5728-4037-9a62-09497be781d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:08.266955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755589d8-5728-4037-9a62-09497be781d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:08.267248 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755589d8-5728-4037-9a62-09497be781d6 admin admin] PATH_INFO: `/v3/projects/a24f5f4930bf4726910dd1c12dadc83a/users/e7db45e6d45a47baabf130d4e039681f/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:08.304519 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755589d8-5728-4037-9a62-09497be781d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a24f5f4930bf4726910dd1c12dadc83a, user_id=e7db45e6d45a47baabf130d4e039681f, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:08.310576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755589d8-5728-4037-9a62-09497be781d6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:08.327370 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 963/1942] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:08 2022] PUT /identity/v3/projects/a24f5f4930bf4726910dd1c12dadc83a/users/e7db45e6d45a47baabf130d4e039681f/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:08.340743 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747448f-2877-4063-aaa9-1682293d01bb None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:08.341064 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747448f-2877-4063-aaa9-1682293d01bb None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:08.341365 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747448f-2877-4063-aaa9-1682293d01bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:08.374018 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-4747448f-2877-4063-aaa9-1682293d01bb None None] MFA Rules not processed for user `e7db45e6d45a47baabf130d4e039681f`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:08.376490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4747448f-2877-4063-aaa9-1682293d01bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:08.456270 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 980/1943] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:08 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:08.469600 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755589d8-5728-4037-9a62-09497be781d6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:08.471422 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-755589d8-5728-4037-9a62-09497be781d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:08.526735 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755589d8-5728-4037-9a62-09497be781d6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:08.530881 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e840d-8764-4f5e-b689-c72755c7ad3d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6180393da0f495db964f0b2588d8b90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:08.531574 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e840d-8764-4f5e-b689-c72755c7ad3d service glance] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:08.531860 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e840d-8764-4f5e-b689-c72755c7ad3d service glance] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:08.532139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e840d-8764-4f5e-b689-c72755c7ad3d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:08.540303 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e840d-8764-4f5e-b689-c72755c7ad3d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:08.541727 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e840d-8764-4f5e-b689-c72755c7ad3d service glance] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:08.550397 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 964/1944] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:08 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:14.884596 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747448f-2877-4063-aaa9-1682293d01bb None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:14.886770 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4747448f-2877-4063-aaa9-1682293d01bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:14.955549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:14.985578 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747448f-2877-4063-aaa9-1682293d01bb None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:14.993396 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcff6af-be5e-4559-9d42-e820ed92d908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:14.994367 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcff6af-be5e-4559-9d42-e820ed92d908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:14.994711 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcff6af-be5e-4559-9d42-e820ed92d908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:14.995197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcff6af-be5e-4559-9d42-e820ed92d908 admin admin] PATH_INFO: `/v3/users/5b287783bb2c4380b9a122da75a25da8` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:14.996867 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcff6af-be5e-4559-9d42-e820ed92d908 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b287783bb2c4380b9a122da75a25da8)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:15.000225 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcff6af-be5e-4559-9d42-e820ed92d908 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:15.085910 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-bdcff6af-be5e-4559-9d42-e820ed92d908 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b287783bb2c4380b9a122da75a25da8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:15.101131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-bdcff6af-be5e-4559-9d42-e820ed92d908 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b287783bb2c4380b9a122da75a25da8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:15.102047 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-bdcff6af-be5e-4559-9d42-e820ed92d908 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b287783bb2c4380b9a122da75a25da8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:15.105839 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-bdcff6af-be5e-4559-9d42-e820ed92d908 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b287783bb2c4380b9a122da75a25da8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:15.118771 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-bdcff6af-be5e-4559-9d42-e820ed92d908 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b287783bb2c4380b9a122da75a25da8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:15.137844 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 981/1945] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:14 2022] DELETE /identity/v3/users/5b287783bb2c4380b9a122da75a25da8 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:15.148300 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62b766-bb4c-4210-8d58-b75440361fb2 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:15.148300 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62b766-bb4c-4210-8d58-b75440361fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:15.148300 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62b766-bb4c-4210-8d58-b75440361fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:15.178521 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8d62b766-bb4c-4210-8d58-b75440361fb2 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:15.179975 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8d62b766-bb4c-4210-8d58-b75440361fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:15.301292 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 965/1946] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:15.308428 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcff6af-be5e-4559-9d42-e820ed92d908 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:15.309902 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-bdcff6af-be5e-4559-9d42-e820ed92d908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:15.408177 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcff6af-be5e-4559-9d42-e820ed92d908 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:15.411965 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4b90f-112d-486d-87c2-7854d0ff8ba4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:15.412963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4b90f-112d-486d-87c2-7854d0ff8ba4 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:15.413270 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4b90f-112d-486d-87c2-7854d0ff8ba4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:15.413552 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4b90f-112d-486d-87c2-7854d0ff8ba4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:15.425106 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a4b90f-112d-486d-87c2-7854d0ff8ba4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:15.426572 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a4b90f-112d-486d-87c2-7854d0ff8ba4 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:15.440841 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 982/1947] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:22:15.821481 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62b766-bb4c-4210-8d58-b75440361fb2 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:15.823502 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8d62b766-bb4c-4210-8d58-b75440361fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:15.856156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62b766-bb4c-4210-8d58-b75440361fb2 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:15.862770 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717716d7-9bb8-4723-80ef-668ef90ff9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:15.863710 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717716d7-9bb8-4723-80ef-668ef90ff9a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:15.864043 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717716d7-9bb8-4723-80ef-668ef90ff9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:15.864482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717716d7-9bb8-4723-80ef-668ef90ff9a8 admin admin] PATH_INFO: `/v3/projects/63a4ece4819f4abc8e66fc7c1e209154` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:15.866032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717716d7-9bb8-4723-80ef-668ef90ff9a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63a4ece4819f4abc8e66fc7c1e209154)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:15.868640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717716d7-9bb8-4723-80ef-668ef90ff9a8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:15.898182 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:15.933255 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-717716d7-9bb8-4723-80ef-668ef90ff9a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63a4ece4819f4abc8e66fc7c1e209154', 'request_id': 'req-717716d7-9bb8-4723-80ef-668ef90ff9a8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:15.944922 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-717716d7-9bb8-4723-80ef-668ef90ff9a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63a4ece4819f4abc8e66fc7c1e209154', 'request_id': 'req-717716d7-9bb8-4723-80ef-668ef90ff9a8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:15.961421 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-717716d7-9bb8-4723-80ef-668ef90ff9a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63a4ece4819f4abc8e66fc7c1e209154', 'request_id': 'req-717716d7-9bb8-4723-80ef-668ef90ff9a8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:15.980131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-717716d7-9bb8-4723-80ef-668ef90ff9a8 admin admin] The token cache is being invalidate because project 63a4ece4819f4abc8e66fc7c1e209154 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:15.981398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-717716d7-9bb8-4723-80ef-668ef90ff9a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:15.984311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-717716d7-9bb8-4723-80ef-668ef90ff9a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:15.987408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 966/1948] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:15 2022] DELETE /identity/v3/projects/63a4ece4819f4abc8e66fc7c1e209154 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:15.996802 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676048f8-a6ca-45fc-aa97-4b766ea79026 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:15.997366 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676048f8-a6ca-45fc-aa97-4b766ea79026 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:15.997366 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676048f8-a6ca-45fc-aa97-4b766ea79026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:16.020929 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-676048f8-a6ca-45fc-aa97-4b766ea79026 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:16.022132 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-676048f8-a6ca-45fc-aa97-4b766ea79026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:16.109479 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 983/1949] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:16.118204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6720ce-4e9d-4bf3-bb17-309ac4c2defb None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:16.118483 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6720ce-4e9d-4bf3-bb17-309ac4c2defb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:16.118770 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6720ce-4e9d-4bf3-bb17-309ac4c2defb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:16.166650 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7b6720ce-4e9d-4bf3-bb17-309ac4c2defb None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:16.168322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7b6720ce-4e9d-4bf3-bb17-309ac4c2defb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:16.187566 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 967/1950] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:16.193720 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676048f8-a6ca-45fc-aa97-4b766ea79026 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:16.205068 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676048f8-a6ca-45fc-aa97-4b766ea79026 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:16.209960 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2b6e7-3a09-43d9-9cc3-f055cf9fafa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:16.210659 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2b6e7-3a09-43d9-9cc3-f055cf9fafa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:16.210946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2b6e7-3a09-43d9-9cc3-f055cf9fafa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:16.211273 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2b6e7-3a09-43d9-9cc3-f055cf9fafa5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:16.212236 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2b6e7-3a09-43d9-9cc3-f055cf9fafa5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:22:16.212545 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2b6e7-3a09-43d9-9cc3-f055cf9fafa5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:16.214188 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2b6e7-3a09-43d9-9cc3-f055cf9fafa5 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:16.221660 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:16.227992 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 984/1951] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:22:16.232295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6720ce-4e9d-4bf3-bb17-309ac4c2defb None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:16.243688 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6720ce-4e9d-4bf3-bb17-309ac4c2defb None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:16.249680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46419ff-5470-49fb-a270-c8b6da567dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:16.250513 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46419ff-5470-49fb-a270-c8b6da567dce admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:16.250803 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46419ff-5470-49fb-a270-c8b6da567dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:16.251256 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46419ff-5470-49fb-a270-c8b6da567dce admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:16.252034 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46419ff-5470-49fb-a270-c8b6da567dce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:16.254225 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46419ff-5470-49fb-a270-c8b6da567dce admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:16.288817 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 968/1952] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:16 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:16.291586 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2b6e7-3a09-43d9-9cc3-f055cf9fafa5 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:16.302596 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2b6e7-3a09-43d9-9cc3-f055cf9fafa5 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:16.362652 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1657d4-e9cc-450d-8857-c0a5ca8c01a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:16.363387 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1657d4-e9cc-450d-8857-c0a5ca8c01a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:16.363674 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1657d4-e9cc-450d-8857-c0a5ca8c01a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:16.363951 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1657d4-e9cc-450d-8857-c0a5ca8c01a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:16.364655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1657d4-e9cc-450d-8857-c0a5ca8c01a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:16.366575 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1657d4-e9cc-450d-8857-c0a5ca8c01a7 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:16.398476 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 985/1953] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:16 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:16.402600 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46419ff-5470-49fb-a270-c8b6da567dce admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:16.416995 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46419ff-5470-49fb-a270-c8b6da567dce admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:16.424774 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02efae-dc56-45cb-a5a1-eef0b2d124ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:16.426061 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02efae-dc56-45cb-a5a1-eef0b2d124ec admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:16.426527 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02efae-dc56-45cb-a5a1-eef0b2d124ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:16.427445 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02efae-dc56-45cb-a5a1-eef0b2d124ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:16.428548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d02efae-dc56-45cb-a5a1-eef0b2d124ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:16.430561 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d02efae-dc56-45cb-a5a1-eef0b2d124ec admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:16.459880 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 969/1954] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:16.464277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1657d4-e9cc-450d-8857-c0a5ca8c01a7 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:16.479437 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1657d4-e9cc-450d-8857-c0a5ca8c01a7 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:16.494610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb85ae9-09fe-4456-9919-91aa27bc1b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:16.495786 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb85ae9-09fe-4456-9919-91aa27bc1b2f admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:16.500678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb85ae9-09fe-4456-9919-91aa27bc1b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:16.500977 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb85ae9-09fe-4456-9919-91aa27bc1b2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:16.501992 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb85ae9-09fe-4456-9919-91aa27bc1b2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:16.503551 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb85ae9-09fe-4456-9919-91aa27bc1b2f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:16.532406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 986/1955] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:16.537385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02efae-dc56-45cb-a5a1-eef0b2d124ec admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:16.553919 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02efae-dc56-45cb-a5a1-eef0b2d124ec admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:16.568710 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a839b6-9c9d-4165-a109-26138f222ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:16.570069 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a839b6-9c9d-4165-a109-26138f222ca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:16.570480 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a839b6-9c9d-4165-a109-26138f222ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:16.571290 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a839b6-9c9d-4165-a109-26138f222ca3 admin admin] PATH_INFO: `/v3/projects/438a1d9ddc1748229a377a64cd453792/users/ec9030707c8f4b3cba79a96901cac687/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:16.608806 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a839b6-9c9d-4165-a109-26138f222ca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=438a1d9ddc1748229a377a64cd453792, user_id=ec9030707c8f4b3cba79a96901cac687, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:16.611776 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a839b6-9c9d-4165-a109-26138f222ca3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:16.648379 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 970/1956] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:16 2022] PUT /identity/v3/projects/438a1d9ddc1748229a377a64cd453792/users/ec9030707c8f4b3cba79a96901cac687/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:16.650385 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41519a-5ddd-41e7-8cef-ed4603196ba7 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:16.651328 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41519a-5ddd-41e7-8cef-ed4603196ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:16.651697 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41519a-5ddd-41e7-8cef-ed4603196ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:16.702987 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-1b41519a-5ddd-41e7-8cef-ed4603196ba7 None None] MFA Rules not processed for user `ec9030707c8f4b3cba79a96901cac687`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:16.704543 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1b41519a-5ddd-41e7-8cef-ed4603196ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:16.843134 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 987/1957] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:16.848105 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a839b6-9c9d-4165-a109-26138f222ca3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:16.965923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a839b6-9c9d-4165-a109-26138f222ca3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:16.971470 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6dddc-abbc-4aa0-8051-bcc48a312cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:16.972314 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6dddc-abbc-4aa0-8051-bcc48a312cbc admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:16.972605 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6dddc-abbc-4aa0-8051-bcc48a312cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:16.972916 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6dddc-abbc-4aa0-8051-bcc48a312cbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:16.973663 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb6dddc-abbc-4aa0-8051-bcc48a312cbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:16.977385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb6dddc-abbc-4aa0-8051-bcc48a312cbc admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:17.009860 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 971/1958] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:16 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:17.014928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b41519a-5ddd-41e7-8cef-ed4603196ba7 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:17.029304 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b41519a-5ddd-41e7-8cef-ed4603196ba7 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:17.104288 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac0135-4c37-493d-9779-31785219ba94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:17.105134 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac0135-4c37-493d-9779-31785219ba94 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:17.105422 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac0135-4c37-493d-9779-31785219ba94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:17.105697 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac0135-4c37-493d-9779-31785219ba94 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:17.106586 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac0135-4c37-493d-9779-31785219ba94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:17.110123 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac0135-4c37-493d-9779-31785219ba94 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:17.145534 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:17.172912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6dddc-abbc-4aa0-8051-bcc48a312cbc admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:17.175799 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 988/1959] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:17 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:17.193545 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6dddc-abbc-4aa0-8051-bcc48a312cbc admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:17.209833 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa828fd-7680-49ef-9e8d-503c8d84833d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:17.210433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa828fd-7680-49ef-9e8d-503c8d84833d admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:17.210911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa828fd-7680-49ef-9e8d-503c8d84833d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:17.211320 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa828fd-7680-49ef-9e8d-503c8d84833d admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:17.212522 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa828fd-7680-49ef-9e8d-503c8d84833d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:17.214155 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa828fd-7680-49ef-9e8d-503c8d84833d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:17.229461 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 972/1960] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:17.234435 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac0135-4c37-493d-9779-31785219ba94 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:17.255042 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac0135-4c37-493d-9779-31785219ba94 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:17.261083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b765aeee-52cb-4e87-8137-62b2a526893e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:17.261910 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765aeee-52cb-4e87-8137-62b2a526893e admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:17.262197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765aeee-52cb-4e87-8137-62b2a526893e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:17.262476 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765aeee-52cb-4e87-8137-62b2a526893e admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:17.263341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b765aeee-52cb-4e87-8137-62b2a526893e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:17.265971 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b765aeee-52cb-4e87-8137-62b2a526893e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:17.279152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 989/1961] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:22:17.285759 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa828fd-7680-49ef-9e8d-503c8d84833d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:17.297497 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa828fd-7680-49ef-9e8d-503c8d84833d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:17.302632 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3c577-80f0-496f-a480-1f84f7e1ec3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:17.303942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3c577-80f0-496f-a480-1f84f7e1ec3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:17.304352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3c577-80f0-496f-a480-1f84f7e1ec3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:17.305156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3c577-80f0-496f-a480-1f84f7e1ec3f admin admin] PATH_INFO: `/v3/projects/a0f47622605542f090807f8c2aec0cbe/users/a84ef3cce52e46c7a56f70f234f788e9/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:17.326107 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea3c577-80f0-496f-a480-1f84f7e1ec3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0f47622605542f090807f8c2aec0cbe, user_id=a84ef3cce52e46c7a56f70f234f788e9, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:17.329080 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea3c577-80f0-496f-a480-1f84f7e1ec3f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:17.333824 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:17.353556 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 973/1962] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:17 2022] PUT /identity/v3/projects/a0f47622605542f090807f8c2aec0cbe/users/a84ef3cce52e46c7a56f70f234f788e9/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:17.365569 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b024aa8-aa3e-4278-b22b-f59baaae79c2 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:17.365860 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b024aa8-aa3e-4278-b22b-f59baaae79c2 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:17.366146 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b024aa8-aa3e-4278-b22b-f59baaae79c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:17.394631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-5b024aa8-aa3e-4278-b22b-f59baaae79c2 None None] MFA Rules not processed for user `a84ef3cce52e46c7a56f70f234f788e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:17.395849 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5b024aa8-aa3e-4278-b22b-f59baaae79c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:17.459822 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 990/1963] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:17.464265 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3c577-80f0-496f-a480-1f84f7e1ec3f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:17.537768 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3c577-80f0-496f-a480-1f84f7e1ec3f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:17.542657 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d9ce0-644f-4979-9727-cfb97a027a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:17.543509 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d9ce0-644f-4979-9727-cfb97a027a0b admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:17.543863 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d9ce0-644f-4979-9727-cfb97a027a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:17.544150 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d9ce0-644f-4979-9727-cfb97a027a0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:17.544919 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d9ce0-644f-4979-9727-cfb97a027a0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:17.547617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d9ce0-644f-4979-9727-cfb97a027a0b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:17.596446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 974/1964] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:17 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:17.597861 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b024aa8-aa3e-4278-b22b-f59baaae79c2 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:17.612227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b024aa8-aa3e-4278-b22b-f59baaae79c2 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:17.650082 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d9ce0-644f-4979-9727-cfb97a027a0b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:17.651871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-768d9ce0-644f-4979-9727-cfb97a027a0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:17.678972 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d192c-4388-4f5f-92b9-a4836b6f7ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:17.679675 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d192c-4388-4f5f-92b9-a4836b6f7ee0 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:17.679957 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d192c-4388-4f5f-92b9-a4836b6f7ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:17.680247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d192c-4388-4f5f-92b9-a4836b6f7ee0 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:17.680949 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d192c-4388-4f5f-92b9-a4836b6f7ee0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:17.682853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d192c-4388-4f5f-92b9-a4836b6f7ee0 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:17.704259 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 991/1965] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:17 2022] POST /identity/v3/users => generated 300 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:17.708201 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d192c-4388-4f5f-92b9-a4836b6f7ee0 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:17.718310 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d192c-4388-4f5f-92b9-a4836b6f7ee0 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:17.723914 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777f05d-def2-4378-b718-7ea68be6e70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:17.724071 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d9ce0-644f-4979-9727-cfb97a027a0b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:17.724583 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777f05d-def2-4378-b718-7ea68be6e70d admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:17.724929 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777f05d-def2-4378-b718-7ea68be6e70d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:17.725515 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777f05d-def2-4378-b718-7ea68be6e70d admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:17.726326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a777f05d-def2-4378-b718-7ea68be6e70d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:17.727945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a777f05d-def2-4378-b718-7ea68be6e70d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:17.729853 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bf8c7-36cc-45b9-b183-641e8c06ea97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:17.730723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bf8c7-36cc-45b9-b183-641e8c06ea97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:17.731001 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bf8c7-36cc-45b9-b183-641e8c06ea97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:17.731302 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bf8c7-36cc-45b9-b183-641e8c06ea97 admin admin] PATH_INFO: `/v3/users/d8821c3b0f394f64909bdd9a46b407e0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:17.732545 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450bf8c7-36cc-45b9-b183-641e8c06ea97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8821c3b0f394f64909bdd9a46b407e0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:17.734735 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450bf8c7-36cc-45b9-b183-641e8c06ea97 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:17.738090 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 992/1966] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:22:17.742224 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777f05d-def2-4378-b718-7ea68be6e70d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:17.753594 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777f05d-def2-4378-b718-7ea68be6e70d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:17.758604 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474caca-3815-4b56-9241-9e0634a9c072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:17.759734 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474caca-3815-4b56-9241-9e0634a9c072 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:17.760022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474caca-3815-4b56-9241-9e0634a9c072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:17.760299 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474caca-3815-4b56-9241-9e0634a9c072 admin admin] PATH_INFO: `/v3/projects/9f7031aacc50426cbdedeebcf6016e8c/users/97aefb29de8a429aa0c06a843825ab52/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:17.777328 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8474caca-3815-4b56-9241-9e0634a9c072 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f7031aacc50426cbdedeebcf6016e8c, user_id=97aefb29de8a429aa0c06a843825ab52, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:17.780996 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8474caca-3815-4b56-9241-9e0634a9c072 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:17.785981 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-450bf8c7-36cc-45b9-b183-641e8c06ea97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8821c3b0f394f64909bdd9a46b407e0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:17.786694 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-450bf8c7-36cc-45b9-b183-641e8c06ea97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8821c3b0f394f64909bdd9a46b407e0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:17.787397 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-450bf8c7-36cc-45b9-b183-641e8c06ea97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8821c3b0f394f64909bdd9a46b407e0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:17.794589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-450bf8c7-36cc-45b9-b183-641e8c06ea97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8821c3b0f394f64909bdd9a46b407e0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:17.795051 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 993/1967] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:17 2022] PUT /identity/v3/projects/9f7031aacc50426cbdedeebcf6016e8c/users/97aefb29de8a429aa0c06a843825ab52/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:22:17.798765 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474caca-3815-4b56-9241-9e0634a9c072 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:17.800597 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8474caca-3815-4b56-9241-9e0634a9c072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:17.806652 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-450bf8c7-36cc-45b9-b183-641e8c06ea97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8821c3b0f394f64909bdd9a46b407e0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:17.816827 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:17.823147 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 975/1968] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:17 2022] DELETE /identity/v3/users/d8821c3b0f394f64909bdd9a46b407e0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:17.828451 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00171fb-5b99-4aa4-b413-ac7ff9f055f3 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:17.828780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00171fb-5b99-4aa4-b413-ac7ff9f055f3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:17.829004 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00171fb-5b99-4aa4-b413-ac7ff9f055f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:17.833673 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:17.848027 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a00171fb-5b99-4aa4-b413-ac7ff9f055f3 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:17.849226 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a00171fb-5b99-4aa4-b413-ac7ff9f055f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:17.874963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474caca-3815-4b56-9241-9e0634a9c072 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:17.915573 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 976/1969] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:17.925183 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00171fb-5b99-4aa4-b413-ac7ff9f055f3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:17.926516 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a00171fb-5b99-4aa4-b413-ac7ff9f055f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:17.933401 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e3a922-35b9-47bb-9c2f-baa102963431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:17.934191 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e3a922-35b9-47bb-9c2f-baa102963431 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:17.934575 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e3a922-35b9-47bb-9c2f-baa102963431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:17.934949 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e3a922-35b9-47bb-9c2f-baa102963431 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:17.936368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e3a922-35b9-47bb-9c2f-baa102963431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:17.938075 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e3a922-35b9-47bb-9c2f-baa102963431 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:17.950916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 994/1970] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:22:17.953213 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e3a922-35b9-47bb-9c2f-baa102963431 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:17.966642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e3a922-35b9-47bb-9c2f-baa102963431 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:17.974048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634345a-2e1e-4d0d-b7d2-3ec889a1f648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:17.976435 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634345a-2e1e-4d0d-b7d2-3ec889a1f648 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:17.976734 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634345a-2e1e-4d0d-b7d2-3ec889a1f648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:17.977102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634345a-2e1e-4d0d-b7d2-3ec889a1f648 admin admin] PATH_INFO: `/v3/projects/9f7031aacc50426cbdedeebcf6016e8c/users/97aefb29de8a429aa0c06a843825ab52/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:17.981622 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3634345a-2e1e-4d0d-b7d2-3ec889a1f648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f7031aacc50426cbdedeebcf6016e8c, user_id=97aefb29de8a429aa0c06a843825ab52, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:17.984434 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3634345a-2e1e-4d0d-b7d2-3ec889a1f648 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:18.009000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 995/1971] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:17 2022] PUT /identity/v3/projects/9f7031aacc50426cbdedeebcf6016e8c/users/97aefb29de8a429aa0c06a843825ab52/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:22:18.018369 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00171fb-5b99-4aa4-b413-ac7ff9f055f3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:18.021457 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5a97c-9473-40e8-a7c0-33cc554248ec None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:18.022263 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5a97c-9473-40e8-a7c0-33cc554248ec None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:18.022592 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5a97c-9473-40e8-a7c0-33cc554248ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:18.054380 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-89c5a97c-9473-40e8-a7c0-33cc554248ec None None] MFA Rules not processed for user `97aefb29de8a429aa0c06a843825ab52`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:18.055718 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-89c5a97c-9473-40e8-a7c0-33cc554248ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:18.089309 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f1e40-dfed-4bf4-a3f1-5490cefa6cb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:18.089998 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f1e40-dfed-4bf4-a3f1-5490cefa6cb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:18.090281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f1e40-dfed-4bf4-a3f1-5490cefa6cb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:18.090559 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f1e40-dfed-4bf4-a3f1-5490cefa6cb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:18.150408 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 996/1972] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:22:18.174839 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5a97c-9473-40e8-a7c0-33cc554248ec None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:18.176696 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-89c5a97c-9473-40e8-a7c0-33cc554248ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:18.203307 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f1e40-dfed-4bf4-a3f1-5490cefa6cb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:18.204768 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f1e40-dfed-4bf4-a3f1-5490cefa6cb8 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:18.215991 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 977/1973] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:22:18.241060 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:18.258361 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5a97c-9473-40e8-a7c0-33cc554248ec None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:18.266677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93de928-d97c-4670-b9aa-9ee8abb99227 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:18.266677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93de928-d97c-4670-b9aa-9ee8abb99227 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:18.266677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93de928-d97c-4670-b9aa-9ee8abb99227 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:18.266677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93de928-d97c-4670-b9aa-9ee8abb99227 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:18.281404 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93de928-d97c-4670-b9aa-9ee8abb99227 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:18.282934 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93de928-d97c-4670-b9aa-9ee8abb99227 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:18.294585 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 997/1974] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:18 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:18.731690 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f1e40-dfed-4bf4-a3f1-5490cefa6cb8 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:18.734261 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3c0f1e40-dfed-4bf4-a3f1-5490cefa6cb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:18.758418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f1e40-dfed-4bf4-a3f1-5490cefa6cb8 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:18.767605 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8295a2d-41cc-4fc4-9375-3cf390a5936f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:18.769332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8295a2d-41cc-4fc4-9375-3cf390a5936f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:18.769759 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8295a2d-41cc-4fc4-9375-3cf390a5936f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:18.770180 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8295a2d-41cc-4fc4-9375-3cf390a5936f admin admin] PATH_INFO: `/v3/projects/4f6b7f56d4f24f13b3b935a9bf0e6efc` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:18.772248 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8295a2d-41cc-4fc4-9375-3cf390a5936f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f6b7f56d4f24f13b3b935a9bf0e6efc)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:18.774859 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8295a2d-41cc-4fc4-9375-3cf390a5936f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:18.777906 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:18.878813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f8295a2d-41cc-4fc4-9375-3cf390a5936f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f6b7f56d4f24f13b3b935a9bf0e6efc', 'request_id': 'req-f8295a2d-41cc-4fc4-9375-3cf390a5936f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:18.899904 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f8295a2d-41cc-4fc4-9375-3cf390a5936f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f6b7f56d4f24f13b3b935a9bf0e6efc', 'request_id': 'req-f8295a2d-41cc-4fc4-9375-3cf390a5936f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:18.907996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f8295a2d-41cc-4fc4-9375-3cf390a5936f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f6b7f56d4f24f13b3b935a9bf0e6efc', 'request_id': 'req-f8295a2d-41cc-4fc4-9375-3cf390a5936f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:18.929143 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f8295a2d-41cc-4fc4-9375-3cf390a5936f admin admin] The token cache is being invalidate because project 4f6b7f56d4f24f13b3b935a9bf0e6efc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:18.929344 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f8295a2d-41cc-4fc4-9375-3cf390a5936f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:18.930520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f8295a2d-41cc-4fc4-9375-3cf390a5936f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:18.935108 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 978/1975] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:18 2022] DELETE /identity/v3/projects/4f6b7f56d4f24f13b3b935a9bf0e6efc => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:18.937309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93de928-d97c-4670-b9aa-9ee8abb99227 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:18.954361 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93de928-d97c-4670-b9aa-9ee8abb99227 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:19.054253 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abaf311-6d94-48e7-b8ce-e7178e773cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:19.055085 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abaf311-6d94-48e7-b8ce-e7178e773cf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:19.055410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abaf311-6d94-48e7-b8ce-e7178e773cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:19.055819 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abaf311-6d94-48e7-b8ce-e7178e773cf9 admin admin] PATH_INFO: `/v3/users/108f9071ce904e8fb10997ceb84c5857` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:19.057157 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abaf311-6d94-48e7-b8ce-e7178e773cf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=108f9071ce904e8fb10997ceb84c5857)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:19.059174 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abaf311-6d94-48e7-b8ce-e7178e773cf9 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:19.152073 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8295a2d-41cc-4fc4-9375-3cf390a5936f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:19.156059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f8295a2d-41cc-4fc4-9375-3cf390a5936f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:19.165210 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3abaf311-6d94-48e7-b8ce-e7178e773cf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '108f9071ce904e8fb10997ceb84c5857'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:19.187415 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3abaf311-6d94-48e7-b8ce-e7178e773cf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '108f9071ce904e8fb10997ceb84c5857'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:19.188314 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3abaf311-6d94-48e7-b8ce-e7178e773cf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '108f9071ce904e8fb10997ceb84c5857'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:19.189033 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3abaf311-6d94-48e7-b8ce-e7178e773cf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '108f9071ce904e8fb10997ceb84c5857'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:19.196704 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3abaf311-6d94-48e7-b8ce-e7178e773cf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '108f9071ce904e8fb10997ceb84c5857'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:19.218868 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 998/1976] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:18 2022] DELETE /identity/v3/users/108f9071ce904e8fb10997ceb84c5857 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:19.251778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8295a2d-41cc-4fc4-9375-3cf390a5936f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:19.303333 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb552f48-e970-4d0f-95f7-f17aaf0655da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:19.303998 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb552f48-e970-4d0f-95f7-f17aaf0655da service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:19.304283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb552f48-e970-4d0f-95f7-f17aaf0655da service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:19.304560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb552f48-e970-4d0f-95f7-f17aaf0655da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:19.306481 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-eb552f48-e970-4d0f-95f7-f17aaf0655da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:19.381825 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb552f48-e970-4d0f-95f7-f17aaf0655da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:19.383526 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb552f48-e970-4d0f-95f7-f17aaf0655da service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:19.389295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:19.395864 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 979/1977] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:22:19.750322 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abaf311-6d94-48e7-b8ce-e7178e773cf9 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:19.752040 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3abaf311-6d94-48e7-b8ce-e7178e773cf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:19.850258 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abaf311-6d94-48e7-b8ce-e7178e773cf9 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:19.871090 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f44c6-e158-4cf7-acea-1785c76c2af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:19.874937 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f44c6-e158-4cf7-acea-1785c76c2af7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:19.874937 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f44c6-e158-4cf7-acea-1785c76c2af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:19.874937 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f44c6-e158-4cf7-acea-1785c76c2af7 admin admin] PATH_INFO: `/v3/projects/85df0f73285b4d2fb84b0d6305996fbd` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:19.880164 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f44c6-e158-4cf7-acea-1785c76c2af7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85df0f73285b4d2fb84b0d6305996fbd)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:19.898015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f44c6-e158-4cf7-acea-1785c76c2af7 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:19.920055 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:19.974253 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cd4f44c6-e158-4cf7-acea-1785c76c2af7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85df0f73285b4d2fb84b0d6305996fbd', 'request_id': 'req-cd4f44c6-e158-4cf7-acea-1785c76c2af7'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:19.983641 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cd4f44c6-e158-4cf7-acea-1785c76c2af7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85df0f73285b4d2fb84b0d6305996fbd', 'request_id': 'req-cd4f44c6-e158-4cf7-acea-1785c76c2af7'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:19.991183 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cd4f44c6-e158-4cf7-acea-1785c76c2af7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85df0f73285b4d2fb84b0d6305996fbd', 'request_id': 'req-cd4f44c6-e158-4cf7-acea-1785c76c2af7'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:20.002691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cd4f44c6-e158-4cf7-acea-1785c76c2af7 admin admin] The token cache is being invalidate because project 85df0f73285b4d2fb84b0d6305996fbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:20.002986 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cd4f44c6-e158-4cf7-acea-1785c76c2af7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:20.003791 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cd4f44c6-e158-4cf7-acea-1785c76c2af7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:20.005387 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 999/1978] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:19 2022] DELETE /identity/v3/projects/85df0f73285b4d2fb84b0d6305996fbd => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:20.010738 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e400dbd-3b4b-4f0d-bcbd-81ce1699c23f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:20.010974 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e400dbd-3b4b-4f0d-bcbd-81ce1699c23f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:20.011446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e400dbd-3b4b-4f0d-bcbd-81ce1699c23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:20.032627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5e400dbd-3b4b-4f0d-bcbd-81ce1699c23f None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:20.033953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5e400dbd-3b4b-4f0d-bcbd-81ce1699c23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:20.093855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f44c6-e158-4cf7-acea-1785c76c2af7 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:20.095913 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-cd4f44c6-e158-4cf7-acea-1785c76c2af7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:20.143139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 980/1979] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:20.158037 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73048f95-2d01-4cc3-ba90-ca333dfa270f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:20.158037 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73048f95-2d01-4cc3-ba90-ca333dfa270f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:20.158037 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73048f95-2d01-4cc3-ba90-ca333dfa270f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:20.192771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-73048f95-2d01-4cc3-ba90-ca333dfa270f None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:20.195063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-73048f95-2d01-4cc3-ba90-ca333dfa270f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:20.208029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 981/1980] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:20.209261 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73048f95-2d01-4cc3-ba90-ca333dfa270f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:20.219373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73048f95-2d01-4cc3-ba90-ca333dfa270f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:20.224077 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77cfbd-e412-489b-a812-e93126f47ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:20.224739 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77cfbd-e412-489b-a812-e93126f47ccd admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:20.225277 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77cfbd-e412-489b-a812-e93126f47ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:20.225551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77cfbd-e412-489b-a812-e93126f47ccd admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:20.226307 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77cfbd-e412-489b-a812-e93126f47ccd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:22:20.226610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77cfbd-e412-489b-a812-e93126f47ccd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:20.229509 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77cfbd-e412-489b-a812-e93126f47ccd admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:20.235280 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f44c6-e158-4cf7-acea-1785c76c2af7 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:20.239851 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b274e-f356-4ada-b453-4e30d346c2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:20.240717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b274e-f356-4ada-b453-4e30d346c2c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:20.240998 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b274e-f356-4ada-b453-4e30d346c2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:20.241270 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b274e-f356-4ada-b453-4e30d346c2c5 admin admin] PATH_INFO: `/v3/users/212ebdcd4c584bebaaa83d814d8069b9` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:20.242195 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 982/1981] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:22:20.242488 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b274e-f356-4ada-b453-4e30d346c2c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=212ebdcd4c584bebaaa83d814d8069b9)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:20.244463 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b274e-f356-4ada-b453-4e30d346c2c5 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:20.246442 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77cfbd-e412-489b-a812-e93126f47ccd admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:20.256815 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77cfbd-e412-489b-a812-e93126f47ccd admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:20.262506 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07bf87-477f-45e8-bbf2-06c60d50decf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:20.263151 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07bf87-477f-45e8-bbf2-06c60d50decf admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:20.263487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07bf87-477f-45e8-bbf2-06c60d50decf admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:20.263872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07bf87-477f-45e8-bbf2-06c60d50decf admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:20.264615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c07bf87-477f-45e8-bbf2-06c60d50decf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:20.266762 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c07bf87-477f-45e8-bbf2-06c60d50decf admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:20.291329 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-287b274e-f356-4ada-b453-4e30d346c2c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '212ebdcd4c584bebaaa83d814d8069b9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:20.293566 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 983/1982] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:20 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:22:20.298389 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07bf87-477f-45e8-bbf2-06c60d50decf admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:20.306941 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-287b274e-f356-4ada-b453-4e30d346c2c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '212ebdcd4c584bebaaa83d814d8069b9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:20.307627 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-287b274e-f356-4ada-b453-4e30d346c2c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '212ebdcd4c584bebaaa83d814d8069b9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:20.308359 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-287b274e-f356-4ada-b453-4e30d346c2c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '212ebdcd4c584bebaaa83d814d8069b9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:20.315732 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-287b274e-f356-4ada-b453-4e30d346c2c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '212ebdcd4c584bebaaa83d814d8069b9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:20.329171 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1000/1983] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:20 2022] DELETE /identity/v3/users/212ebdcd4c584bebaaa83d814d8069b9 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:20.370538 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07bf87-477f-45e8-bbf2-06c60d50decf admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:20.434764 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715b2ae-2875-4051-9ef6-fba024ee6401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:20.436061 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715b2ae-2875-4051-9ef6-fba024ee6401 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:20.436350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715b2ae-2875-4051-9ef6-fba024ee6401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:20.436713 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715b2ae-2875-4051-9ef6-fba024ee6401 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:20.437430 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715b2ae-2875-4051-9ef6-fba024ee6401 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:20.439442 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715b2ae-2875-4051-9ef6-fba024ee6401 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:20.449629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:20.463687 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 984/1984] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:20 2022] POST /identity/v3/users => generated 294 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:20.468101 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b274e-f356-4ada-b453-4e30d346c2c5 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:20.469515 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-287b274e-f356-4ada-b453-4e30d346c2c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:20.484689 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b274e-f356-4ada-b453-4e30d346c2c5 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:20.489403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90cce81-6e03-4ae5-8fa3-e97f0b460bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:20.490084 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cce81-6e03-4ae5-8fa3-e97f0b460bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:20.490368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cce81-6e03-4ae5-8fa3-e97f0b460bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:20.490643 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cce81-6e03-4ae5-8fa3-e97f0b460bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:20.491412 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90cce81-6e03-4ae5-8fa3-e97f0b460bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:20.492995 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90cce81-6e03-4ae5-8fa3-e97f0b460bd8 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:20.504206 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1001/1985] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:20.508219 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715b2ae-2875-4051-9ef6-fba024ee6401 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:20.520581 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715b2ae-2875-4051-9ef6-fba024ee6401 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:20.525308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8f69c-2344-4b6c-aca4-3da39deec46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:20.526622 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8f69c-2344-4b6c-aca4-3da39deec46b admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:20.526912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8f69c-2344-4b6c-aca4-3da39deec46b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:20.527191 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8f69c-2344-4b6c-aca4-3da39deec46b admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:20.527969 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f8f69c-2344-4b6c-aca4-3da39deec46b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:20.529616 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f8f69c-2344-4b6c-aca4-3da39deec46b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:20.545628 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 985/1986] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:22:20.550355 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90cce81-6e03-4ae5-8fa3-e97f0b460bd8 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:20.557903 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:20.559055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8f69c-2344-4b6c-aca4-3da39deec46b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:20.562374 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a7f8f69c-2344-4b6c-aca4-3da39deec46b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:20.563673 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90cce81-6e03-4ae5-8fa3-e97f0b460bd8 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:20.574156 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3fe21-cded-4792-9d13-b87e628439b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:20.575801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3fe21-cded-4792-9d13-b87e628439b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:20.577924 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3fe21-cded-4792-9d13-b87e628439b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:20.578401 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3fe21-cded-4792-9d13-b87e628439b1 admin admin] PATH_INFO: `/v3/projects/61918c19b9624bc788fd472993e47736/users/ed884cad4c804beea3a9875a4c5d5fab/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:20.580994 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8f69c-2344-4b6c-aca4-3da39deec46b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:20.591195 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace689a7-dbff-4cfb-8a3d-dc4efce9666e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:20.594069 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace689a7-dbff-4cfb-8a3d-dc4efce9666e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:20.594801 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace689a7-dbff-4cfb-8a3d-dc4efce9666e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:20.595366 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace689a7-dbff-4cfb-8a3d-dc4efce9666e admin admin] PATH_INFO: `/v3/projects/2a8e39e02778437dbc094914785b7681` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:20.597032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace689a7-dbff-4cfb-8a3d-dc4efce9666e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a8e39e02778437dbc094914785b7681)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:20.599039 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3fe21-cded-4792-9d13-b87e628439b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61918c19b9624bc788fd472993e47736, user_id=ed884cad4c804beea3a9875a4c5d5fab, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:20.603258 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3fe21-cded-4792-9d13-b87e628439b1 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:20.605618 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace689a7-dbff-4cfb-8a3d-dc4efce9666e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:20.622508 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1002/1987] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:20 2022] PUT /identity/v3/projects/61918c19b9624bc788fd472993e47736/users/ed884cad4c804beea3a9875a4c5d5fab/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:20.632809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2183f-8b99-4ddf-a8fc-2adef9efdcf6 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:20.633164 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2183f-8b99-4ddf-a8fc-2adef9efdcf6 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:20.633524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2183f-8b99-4ddf-a8fc-2adef9efdcf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:20.666988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a6a2183f-8b99-4ddf-a8fc-2adef9efdcf6 None None] MFA Rules not processed for user `ed884cad4c804beea3a9875a4c5d5fab`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:20.668946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a6a2183f-8b99-4ddf-a8fc-2adef9efdcf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:20.686676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ace689a7-dbff-4cfb-8a3d-dc4efce9666e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a8e39e02778437dbc094914785b7681', 'request_id': 'req-ace689a7-dbff-4cfb-8a3d-dc4efce9666e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:20.703152 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ace689a7-dbff-4cfb-8a3d-dc4efce9666e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a8e39e02778437dbc094914785b7681', 'request_id': 'req-ace689a7-dbff-4cfb-8a3d-dc4efce9666e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:20.719315 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ace689a7-dbff-4cfb-8a3d-dc4efce9666e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a8e39e02778437dbc094914785b7681', 'request_id': 'req-ace689a7-dbff-4cfb-8a3d-dc4efce9666e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:20.736497 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ace689a7-dbff-4cfb-8a3d-dc4efce9666e admin admin] The token cache is being invalidate because project 2a8e39e02778437dbc094914785b7681 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:20.736792 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ace689a7-dbff-4cfb-8a3d-dc4efce9666e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:20.737482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ace689a7-dbff-4cfb-8a3d-dc4efce9666e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:20.739313 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 986/1988] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:20 2022] DELETE /identity/v3/projects/2a8e39e02778437dbc094914785b7681 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:20.742809 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace689a7-dbff-4cfb-8a3d-dc4efce9666e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:20.744179 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ace689a7-dbff-4cfb-8a3d-dc4efce9666e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:20.775746 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1003/1989] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:22:20.780768 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2183f-8b99-4ddf-a8fc-2adef9efdcf6 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:20.782113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a6a2183f-8b99-4ddf-a8fc-2adef9efdcf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:20.823096 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace689a7-dbff-4cfb-8a3d-dc4efce9666e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:20.828961 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c95e570-85bc-4722-83c8-8bd5e7910b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:20.829856 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95e570-85bc-4722-83c8-8bd5e7910b0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:20.830136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95e570-85bc-4722-83c8-8bd5e7910b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:20.830404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95e570-85bc-4722-83c8-8bd5e7910b0d admin admin] PATH_INFO: `/v3/users/e0af5a5eb22f4eb28415c40abb03d8ed` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:20.832004 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c95e570-85bc-4722-83c8-8bd5e7910b0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0af5a5eb22f4eb28415c40abb03d8ed)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:20.833946 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c95e570-85bc-4722-83c8-8bd5e7910b0d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:20.868976 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2183f-8b99-4ddf-a8fc-2adef9efdcf6 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:20.875145 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d62aa-42a2-447a-9f9d-991819735703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:20.875923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d62aa-42a2-447a-9f9d-991819735703 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:20.876206 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d62aa-42a2-447a-9f9d-991819735703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:20.876488 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d62aa-42a2-447a-9f9d-991819735703 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:20.877229 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d62aa-42a2-447a-9f9d-991819735703 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:20.879425 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d62aa-42a2-447a-9f9d-991819735703 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:20.899497 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0c95e570-85bc-4722-83c8-8bd5e7910b0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0af5a5eb22f4eb28415c40abb03d8ed'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:20.900550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0c95e570-85bc-4722-83c8-8bd5e7910b0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0af5a5eb22f4eb28415c40abb03d8ed'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:20.901196 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0c95e570-85bc-4722-83c8-8bd5e7910b0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0af5a5eb22f4eb28415c40abb03d8ed'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:20.915612 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1004/1990] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:20 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:22:20.917993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d62aa-42a2-447a-9f9d-991819735703 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:20.922136 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-473d62aa-42a2-447a-9f9d-991819735703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:20.923502 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0c95e570-85bc-4722-83c8-8bd5e7910b0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0af5a5eb22f4eb28415c40abb03d8ed'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:20.936945 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:20.945555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0c95e570-85bc-4722-83c8-8bd5e7910b0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0af5a5eb22f4eb28415c40abb03d8ed'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:20.983683 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 987/1991] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:20 2022] DELETE /identity/v3/users/e0af5a5eb22f4eb28415c40abb03d8ed => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:22:21.022051 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:21.029937 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d62aa-42a2-447a-9f9d-991819735703 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:21.121407 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9821316-fe50-4ad4-ac2c-7c7a8b05ca48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:21.122331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9821316-fe50-4ad4-ac2c-7c7a8b05ca48 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:21.122763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9821316-fe50-4ad4-ac2c-7c7a8b05ca48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:21.123920 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9821316-fe50-4ad4-ac2c-7c7a8b05ca48 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:21.124945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9821316-fe50-4ad4-ac2c-7c7a8b05ca48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:21.127124 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9821316-fe50-4ad4-ac2c-7c7a8b05ca48 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:21.170596 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1005/1992] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:20 2022] POST /identity/v3/users => generated 299 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:22:21.171675 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c95e570-85bc-4722-83c8-8bd5e7910b0d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:21.182023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c95e570-85bc-4722-83c8-8bd5e7910b0d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:21.188234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccb465-7a98-45d8-9ac7-be6f7dceec1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:21.188860 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccb465-7a98-45d8-9ac7-be6f7dceec1a admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:21.189141 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccb465-7a98-45d8-9ac7-be6f7dceec1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:21.189410 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccb465-7a98-45d8-9ac7-be6f7dceec1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:21.190138 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ccb465-7a98-45d8-9ac7-be6f7dceec1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:21.191814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ccb465-7a98-45d8-9ac7-be6f7dceec1a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:21.215264 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 988/1993] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:21.219926 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9821316-fe50-4ad4-ac2c-7c7a8b05ca48 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:21.242164 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9821316-fe50-4ad4-ac2c-7c7a8b05ca48 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:21.252948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccb465-7a98-45d8-9ac7-be6f7dceec1a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:21.255862 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-05ccb465-7a98-45d8-9ac7-be6f7dceec1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:21.258295 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e7d8e-d384-4e3d-9c6e-fd952fe1110b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:21.259379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e7d8e-d384-4e3d-9c6e-fd952fe1110b admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:21.260076 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e7d8e-d384-4e3d-9c6e-fd952fe1110b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:21.260076 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e7d8e-d384-4e3d-9c6e-fd952fe1110b admin admin] PATH_INFO: `/v3/projects/dfb39f2c837f483fa6c84c95724dfafc/users/6f7cdbca322447caac481ca1f417f55d/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:21.286433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccb465-7a98-45d8-9ac7-be6f7dceec1a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:21.319864 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d682322-b983-411d-9294-fbb1c8b2d0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:21.321066 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d682322-b983-411d-9294-fbb1c8b2d0b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:21.322454 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d682322-b983-411d-9294-fbb1c8b2d0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:21.322706 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e7d8e-d384-4e3d-9c6e-fd952fe1110b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfb39f2c837f483fa6c84c95724dfafc, user_id=6f7cdbca322447caac481ca1f417f55d, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:21.323099 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d682322-b983-411d-9294-fbb1c8b2d0b6 admin admin] PATH_INFO: `/v3/projects/c224108509ea40bda53e027705732cd4` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:21.327389 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e7d8e-d384-4e3d-9c6e-fd952fe1110b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:21.328947 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d682322-b983-411d-9294-fbb1c8b2d0b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c224108509ea40bda53e027705732cd4)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:21.333942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d682322-b983-411d-9294-fbb1c8b2d0b6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:21.357933 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1006/1994] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:21 2022] PUT /identity/v3/projects/dfb39f2c837f483fa6c84c95724dfafc/users/6f7cdbca322447caac481ca1f417f55d/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:21.358971 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e7d8e-d384-4e3d-9c6e-fd952fe1110b admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:21.424088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0d682322-b983-411d-9294-fbb1c8b2d0b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c224108509ea40bda53e027705732cd4', 'request_id': 'req-0d682322-b983-411d-9294-fbb1c8b2d0b6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:21.441322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0d682322-b983-411d-9294-fbb1c8b2d0b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c224108509ea40bda53e027705732cd4', 'request_id': 'req-0d682322-b983-411d-9294-fbb1c8b2d0b6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:21.449700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0d682322-b983-411d-9294-fbb1c8b2d0b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c224108509ea40bda53e027705732cd4', 'request_id': 'req-0d682322-b983-411d-9294-fbb1c8b2d0b6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:21.472934 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0d682322-b983-411d-9294-fbb1c8b2d0b6 admin admin] The token cache is being invalidate because project c224108509ea40bda53e027705732cd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:21.473412 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0d682322-b983-411d-9294-fbb1c8b2d0b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:21.474398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0d682322-b983-411d-9294-fbb1c8b2d0b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:21.477074 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 989/1995] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:21 2022] DELETE /identity/v3/projects/c224108509ea40bda53e027705732cd4 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:21.484623 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48effa4-535c-47ea-8768-8a3f4501ce94 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:21.485029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48effa4-535c-47ea-8768-8a3f4501ce94 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:21.485313 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48effa4-535c-47ea-8768-8a3f4501ce94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:21.500006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e7d8e-d384-4e3d-9c6e-fd952fe1110b admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:21.508922 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b48effa4-535c-47ea-8768-8a3f4501ce94 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:21.510213 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b48effa4-535c-47ea-8768-8a3f4501ce94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:21.586402 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587e209-fca4-4f4d-a116-ac83bbc2e0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:21.587467 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587e209-fca4-4f4d-a116-ac83bbc2e0e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:21.587825 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587e209-fca4-4f4d-a116-ac83bbc2e0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:21.588193 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587e209-fca4-4f4d-a116-ac83bbc2e0e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:21.588992 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a587e209-fca4-4f4d-a116-ac83bbc2e0e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:21.590922 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a587e209-fca4-4f4d-a116-ac83bbc2e0e9 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:21.603140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1007/1996] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:21.607179 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587e209-fca4-4f4d-a116-ac83bbc2e0e9 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:21.610891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 990/1997] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:22:21.611635 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a587e209-fca4-4f4d-a116-ac83bbc2e0e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:21.618537 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577cffc-3e4e-4e18-8c2e-b5f3086614ed None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:21.618833 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577cffc-3e4e-4e18-8c2e-b5f3086614ed None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:21.619167 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577cffc-3e4e-4e18-8c2e-b5f3086614ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:21.632198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587e209-fca4-4f4d-a116-ac83bbc2e0e9 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:21.641358 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8577cffc-3e4e-4e18-8c2e-b5f3086614ed None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:21.642966 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8577cffc-3e4e-4e18-8c2e-b5f3086614ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:21.646788 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764afe2-d415-4127-b0c4-b3a75c7cc18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:21.648130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764afe2-d415-4127-b0c4-b3a75c7cc18c admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:21.649452 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764afe2-d415-4127-b0c4-b3a75c7cc18c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:21.649967 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764afe2-d415-4127-b0c4-b3a75c7cc18c admin admin] PATH_INFO: `/v3/projects/dfb39f2c837f483fa6c84c95724dfafc/users/6f7cdbca322447caac481ca1f417f55d/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:21.652777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764afe2-d415-4127-b0c4-b3a75c7cc18c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfb39f2c837f483fa6c84c95724dfafc, user_id=6f7cdbca322447caac481ca1f417f55d, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:21.655829 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 991/1998] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:22:21.657451 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764afe2-d415-4127-b0c4-b3a75c7cc18c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:21.658802 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8577cffc-3e4e-4e18-8c2e-b5f3086614ed None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:21.673205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:21.677264 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8577cffc-3e4e-4e18-8c2e-b5f3086614ed None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:21.689339 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1008/1999] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:21 2022] PUT /identity/v3/projects/dfb39f2c837f483fa6c84c95724dfafc/users/6f7cdbca322447caac481ca1f417f55d/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:21.700329 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcff9c-1083-4fa1-982b-4ac21ced9d77 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:21.700545 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcff9c-1083-4fa1-982b-4ac21ced9d77 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:21.700825 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcff9c-1083-4fa1-982b-4ac21ced9d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:21.733191 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-51fcff9c-1083-4fa1-982b-4ac21ced9d77 None None] MFA Rules not processed for user `6f7cdbca322447caac481ca1f417f55d`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:21.734410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-51fcff9c-1083-4fa1-982b-4ac21ced9d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:21.750913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:21.764956 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82cc9c-2194-4bc0-8603-9361a6653ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:21.766258 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82cc9c-2194-4bc0-8603-9361a6653ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:21.766979 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82cc9c-2194-4bc0-8603-9361a6653ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:21.767665 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82cc9c-2194-4bc0-8603-9361a6653ef4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:21.768612 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82cc9c-2194-4bc0-8603-9361a6653ef4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:22:21.769257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82cc9c-2194-4bc0-8603-9361a6653ef4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:21.771147 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82cc9c-2194-4bc0-8603-9361a6653ef4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:21.797123 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 992/2000] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:22:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:22:21.799264 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82cc9c-2194-4bc0-8603-9361a6653ef4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:21.813786 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82cc9c-2194-4bc0-8603-9361a6653ef4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:21.820240 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9916c2b-2b76-4432-a60e-1aa553adae32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:21.821252 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9916c2b-2b76-4432-a60e-1aa553adae32 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:21.821615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9916c2b-2b76-4432-a60e-1aa553adae32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:21.822089 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9916c2b-2b76-4432-a60e-1aa553adae32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:21.823135 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9916c2b-2b76-4432-a60e-1aa553adae32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:21.827013 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9916c2b-2b76-4432-a60e-1aa553adae32 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:21.835732 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1009/2001] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:21 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:21.848154 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fcff9c-1083-4fa1-982b-4ac21ced9d77 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:21.849550 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-51fcff9c-1083-4fa1-982b-4ac21ced9d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:21.865680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 993/2002] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:21 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:21.869953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9916c2b-2b76-4432-a60e-1aa553adae32 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:21.928899 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fcff9c-1083-4fa1-982b-4ac21ced9d77 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:21.952748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9916c2b-2b76-4432-a60e-1aa553adae32 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:21.958962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d55238-d27c-43fc-872a-c400c1a27440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:21.960316 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d55238-d27c-43fc-872a-c400c1a27440 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:21.960706 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d55238-d27c-43fc-872a-c400c1a27440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:21.961375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d55238-d27c-43fc-872a-c400c1a27440 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:21.962206 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d55238-d27c-43fc-872a-c400c1a27440 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:21.965816 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d55238-d27c-43fc-872a-c400c1a27440 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:21.987804 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ad2b8-3f18-48bc-b38d-a2bba7011b1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:21.992973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ad2b8-3f18-48bc-b38d-a2bba7011b1d service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:21.992973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ad2b8-3f18-48bc-b38d-a2bba7011b1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:21.992973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ad2b8-3f18-48bc-b38d-a2bba7011b1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:21.995419 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 994/2003] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:21 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:22:21.997687 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d55238-d27c-43fc-872a-c400c1a27440 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:22.008316 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d55238-d27c-43fc-872a-c400c1a27440 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:22.014434 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd097a-3a65-4901-a09a-7135337fe10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:22.015444 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd097a-3a65-4901-a09a-7135337fe10e admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:22.015753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd097a-3a65-4901-a09a-7135337fe10e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:22.016030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd097a-3a65-4901-a09a-7135337fe10e admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:22.016793 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd097a-3a65-4901-a09a-7135337fe10e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:22.018373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd097a-3a65-4901-a09a-7135337fe10e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:22.034855 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 995/2004] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:22:22.038810 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd097a-3a65-4901-a09a-7135337fe10e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:22.051450 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd097a-3a65-4901-a09a-7135337fe10e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:22.057001 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:22.065760 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564206b8-bc3a-41f8-ac1b-6efe3dfbf2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:22.067090 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564206b8-bc3a-41f8-ac1b-6efe3dfbf2f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:22.067563 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564206b8-bc3a-41f8-ac1b-6efe3dfbf2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:22.068484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564206b8-bc3a-41f8-ac1b-6efe3dfbf2f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:22.069885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564206b8-bc3a-41f8-ac1b-6efe3dfbf2f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:22.072230 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564206b8-bc3a-41f8-ac1b-6efe3dfbf2f5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:22.074792 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ad2b8-3f18-48bc-b38d-a2bba7011b1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:22.077229 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ad2b8-3f18-48bc-b38d-a2bba7011b1d service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:22.087313 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 996/2005] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:22:22.091397 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564206b8-bc3a-41f8-ac1b-6efe3dfbf2f5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:22.092348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1010/2006] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:21 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:22:22.117296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564206b8-bc3a-41f8-ac1b-6efe3dfbf2f5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:22.123024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276bb558-1776-450f-bda4-40efa78cf924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:22.124398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bb558-1776-450f-bda4-40efa78cf924 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:22.124686 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bb558-1776-450f-bda4-40efa78cf924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:22.125127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bb558-1776-450f-bda4-40efa78cf924 admin admin] PATH_INFO: `/v3/projects/8cfdb8a057924628a5a46ad4d5124470/users/140093d4082342e2b4178781f09e643d/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:22.146171 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276bb558-1776-450f-bda4-40efa78cf924 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cfdb8a057924628a5a46ad4d5124470, user_id=140093d4082342e2b4178781f09e643d, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:22.149597 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276bb558-1776-450f-bda4-40efa78cf924 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:22.182605 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ad2b8-3f18-48bc-b38d-a2bba7011b1d service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:22.190431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 997/2007] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:22 2022] PUT /identity/v3/projects/8cfdb8a057924628a5a46ad4d5124470/users/140093d4082342e2b4178781f09e643d/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:22.200218 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ad2b8-3f18-48bc-b38d-a2bba7011b1d service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:22.209804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae219331-f531-4666-b2eb-edd474e327e1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:22.210293 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae219331-f531-4666-b2eb-edd474e327e1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:22.210764 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae219331-f531-4666-b2eb-edd474e327e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:22.253931 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ae219331-f531-4666-b2eb-edd474e327e1 None None] MFA Rules not processed for user `140093d4082342e2b4178781f09e643d`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:22.255486 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ae219331-f531-4666-b2eb-edd474e327e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:22.300496 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de41e940-9978-4bf9-957f-0c52ac25f4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:22.301645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41e940-9978-4bf9-957f-0c52ac25f4f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:22.302066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41e940-9978-4bf9-957f-0c52ac25f4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:22.302350 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41e940-9978-4bf9-957f-0c52ac25f4f8 admin admin] PATH_INFO: `/v3/users/ed884cad4c804beea3a9875a4c5d5fab` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:22.305487 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de41e940-9978-4bf9-957f-0c52ac25f4f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed884cad4c804beea3a9875a4c5d5fab)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:22.307836 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de41e940-9978-4bf9-957f-0c52ac25f4f8 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:22.333649 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:22.355691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 998/2008] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:22.386274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-de41e940-9978-4bf9-957f-0c52ac25f4f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed884cad4c804beea3a9875a4c5d5fab'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:22.406273 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-de41e940-9978-4bf9-957f-0c52ac25f4f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed884cad4c804beea3a9875a4c5d5fab'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:22.407028 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-de41e940-9978-4bf9-957f-0c52ac25f4f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed884cad4c804beea3a9875a4c5d5fab'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:22.407778 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-de41e940-9978-4bf9-957f-0c52ac25f4f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed884cad4c804beea3a9875a4c5d5fab'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:22.416943 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-de41e940-9978-4bf9-957f-0c52ac25f4f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed884cad4c804beea3a9875a4c5d5fab'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:22.437519 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae219331-f531-4666-b2eb-edd474e327e1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:22.438146 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1011/2009] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:22 2022] DELETE /identity/v3/users/ed884cad4c804beea3a9875a4c5d5fab => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:22.442481 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2642fa-bd47-40bc-bffc-557055531db1 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:22.442870 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2642fa-bd47-40bc-bffc-557055531db1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:22.443039 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2642fa-bd47-40bc-bffc-557055531db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:22.463396 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-de2642fa-bd47-40bc-bffc-557055531db1 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:22.464738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-de2642fa-bd47-40bc-bffc-557055531db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:22.550786 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae219331-f531-4666-b2eb-edd474e327e1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:22.598213 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1012/2010] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:22:22.622159 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2642fa-bd47-40bc-bffc-557055531db1 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:22.623953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-de2642fa-bd47-40bc-bffc-557055531db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:22.646040 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:22.699411 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e7204-b6bd-42c7-bebf-8f958bf06ba9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:22.700179 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e7204-b6bd-42c7-bebf-8f958bf06ba9 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:22.700468 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e7204-b6bd-42c7-bebf-8f958bf06ba9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:22.700747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e7204-b6bd-42c7-bebf-8f958bf06ba9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:22.707245 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-464e7204-b6bd-42c7-bebf-8f958bf06ba9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:22.721086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2642fa-bd47-40bc-bffc-557055531db1 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:22.728054 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1527e-3a9b-4da0-ba78-49e19805f54b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:22.728893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1527e-3a9b-4da0-ba78-49e19805f54b service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:22.729818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1527e-3a9b-4da0-ba78-49e19805f54b service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:22.730561 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1527e-3a9b-4da0-ba78-49e19805f54b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:22.743646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1527e-3a9b-4da0-ba78-49e19805f54b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:22.746025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1527e-3a9b-4da0-ba78-49e19805f54b service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:22.768707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1013/2011] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:22.855924 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e7204-b6bd-42c7-bebf-8f958bf06ba9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:22.858297 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e7204-b6bd-42c7-bebf-8f958bf06ba9 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:22.870346 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 999/2012] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:23.499923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1527e-3a9b-4da0-ba78-49e19805f54b service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:23.502231 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-19a1527e-3a9b-4da0-ba78-49e19805f54b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:23.523678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1527e-3a9b-4da0-ba78-49e19805f54b service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:23.535522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232d1ab-7226-45ad-895f-d90b90c8418a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:23.536207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232d1ab-7226-45ad-895f-d90b90c8418a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:23.536500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232d1ab-7226-45ad-895f-d90b90c8418a admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:23.536795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232d1ab-7226-45ad-895f-d90b90c8418a admin admin] PATH_INFO: `/v3/projects/61918c19b9624bc788fd472993e47736` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:23.546906 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9232d1ab-7226-45ad-895f-d90b90c8418a admin admin] RBAC: Authorizing `identity:delete_project(project_id=61918c19b9624bc788fd472993e47736)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:23.549577 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9232d1ab-7226-45ad-895f-d90b90c8418a admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:23.610213 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9232d1ab-7226-45ad-895f-d90b90c8418a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61918c19b9624bc788fd472993e47736', 'request_id': 'req-9232d1ab-7226-45ad-895f-d90b90c8418a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:23.621012 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9232d1ab-7226-45ad-895f-d90b90c8418a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61918c19b9624bc788fd472993e47736', 'request_id': 'req-9232d1ab-7226-45ad-895f-d90b90c8418a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:23.628671 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9232d1ab-7226-45ad-895f-d90b90c8418a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61918c19b9624bc788fd472993e47736', 'request_id': 'req-9232d1ab-7226-45ad-895f-d90b90c8418a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:23.642572 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9232d1ab-7226-45ad-895f-d90b90c8418a admin admin] The token cache is being invalidate because project 61918c19b9624bc788fd472993e47736 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:23.642912 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9232d1ab-7226-45ad-895f-d90b90c8418a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:23.643571 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9232d1ab-7226-45ad-895f-d90b90c8418a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:23.649562 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1014/2013] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:23 2022] DELETE /identity/v3/projects/61918c19b9624bc788fd472993e47736 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:23.651573 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e7204-b6bd-42c7-bebf-8f958bf06ba9 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:23.677194 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e7204-b6bd-42c7-bebf-8f958bf06ba9 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:23.761827 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39962e-dd01-4d23-b99e-8cf524354af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:23.762971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39962e-dd01-4d23-b99e-8cf524354af0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:23.763969 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39962e-dd01-4d23-b99e-8cf524354af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:23.764460 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39962e-dd01-4d23-b99e-8cf524354af0 admin admin] PATH_INFO: `/v3/users/6f7cdbca322447caac481ca1f417f55d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:23.766203 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc39962e-dd01-4d23-b99e-8cf524354af0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f7cdbca322447caac481ca1f417f55d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:23.768373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc39962e-dd01-4d23-b99e-8cf524354af0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:23.784207 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:23.850160 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cc39962e-dd01-4d23-b99e-8cf524354af0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f7cdbca322447caac481ca1f417f55d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:23.851404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cc39962e-dd01-4d23-b99e-8cf524354af0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f7cdbca322447caac481ca1f417f55d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:23.852807 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cc39962e-dd01-4d23-b99e-8cf524354af0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f7cdbca322447caac481ca1f417f55d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:23.880666 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cc39962e-dd01-4d23-b99e-8cf524354af0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f7cdbca322447caac481ca1f417f55d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:23.922290 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cc39962e-dd01-4d23-b99e-8cf524354af0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f7cdbca322447caac481ca1f417f55d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:23.955564 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1000/2014] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:23 2022] DELETE /identity/v3/users/6f7cdbca322447caac481ca1f417f55d => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:23.973515 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232d1ab-7226-45ad-895f-d90b90c8418a admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:23.975359 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9232d1ab-7226-45ad-895f-d90b90c8418a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:24.046865 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232d1ab-7226-45ad-895f-d90b90c8418a admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:24.125529 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca93b190-f538-4856-b648-1fa4dd8f6440 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:24.126217 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93b190-f538-4856-b648-1fa4dd8f6440 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:24.126499 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93b190-f538-4856-b648-1fa4dd8f6440 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:24.127030 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93b190-f538-4856-b648-1fa4dd8f6440 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:24.130760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ca93b190-f538-4856-b648-1fa4dd8f6440 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:24.140081 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:24.231475 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca93b190-f538-4856-b648-1fa4dd8f6440 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:24.232947 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca93b190-f538-4856-b648-1fa4dd8f6440 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:24.242181 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1015/2015] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:24.314515 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39962e-dd01-4d23-b99e-8cf524354af0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:24.317396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cc39962e-dd01-4d23-b99e-8cf524354af0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:24.444002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39962e-dd01-4d23-b99e-8cf524354af0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:24.449414 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323fcee-fe71-4b2b-9de5-f2678eb8ce34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:24.450655 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323fcee-fe71-4b2b-9de5-f2678eb8ce34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:24.451623 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323fcee-fe71-4b2b-9de5-f2678eb8ce34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:24.451623 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323fcee-fe71-4b2b-9de5-f2678eb8ce34 admin admin] PATH_INFO: `/v3/projects/dfb39f2c837f483fa6c84c95724dfafc` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:24.452651 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323fcee-fe71-4b2b-9de5-f2678eb8ce34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfb39f2c837f483fa6c84c95724dfafc)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:24.454838 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323fcee-fe71-4b2b-9de5-f2678eb8ce34 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:24.522308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5323fcee-fe71-4b2b-9de5-f2678eb8ce34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfb39f2c837f483fa6c84c95724dfafc', 'request_id': 'req-5323fcee-fe71-4b2b-9de5-f2678eb8ce34'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:24.534025 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5323fcee-fe71-4b2b-9de5-f2678eb8ce34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfb39f2c837f483fa6c84c95724dfafc', 'request_id': 'req-5323fcee-fe71-4b2b-9de5-f2678eb8ce34'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:24.543951 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5323fcee-fe71-4b2b-9de5-f2678eb8ce34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfb39f2c837f483fa6c84c95724dfafc', 'request_id': 'req-5323fcee-fe71-4b2b-9de5-f2678eb8ce34'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:24.563860 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5323fcee-fe71-4b2b-9de5-f2678eb8ce34 admin admin] The token cache is being invalidate because project dfb39f2c837f483fa6c84c95724dfafc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:24.564429 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5323fcee-fe71-4b2b-9de5-f2678eb8ce34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:24.565369 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5323fcee-fe71-4b2b-9de5-f2678eb8ce34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:24.571283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1001/2016] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:24 2022] DELETE /identity/v3/projects/dfb39f2c837f483fa6c84c95724dfafc => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:24.573509 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf1d2e-b645-4d7b-af3a-4fcdfbf6d650 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:24.573999 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf1d2e-b645-4d7b-af3a-4fcdfbf6d650 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:24.574090 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf1d2e-b645-4d7b-af3a-4fcdfbf6d650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:24.594769 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a5cf1d2e-b645-4d7b-af3a-4fcdfbf6d650 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:24.596455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a5cf1d2e-b645-4d7b-af3a-4fcdfbf6d650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:24.682594 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1016/2017] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:24.691968 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c024e30-9b40-4731-adb9-d442d8c6fcc2 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:24.692452 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c024e30-9b40-4731-adb9-d442d8c6fcc2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:24.692661 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c024e30-9b40-4731-adb9-d442d8c6fcc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:24.725828 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5c024e30-9b40-4731-adb9-d442d8c6fcc2 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:24.727263 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5c024e30-9b40-4731-adb9-d442d8c6fcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:24.736676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1002/2018] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:24.741049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf1d2e-b645-4d7b-af3a-4fcdfbf6d650 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:24.751179 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf1d2e-b645-4d7b-af3a-4fcdfbf6d650 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:24.755955 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8271c-0316-42d0-ad6c-1d9e2b0cd6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:24.756759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8271c-0316-42d0-ad6c-1d9e2b0cd6d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:24.757047 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8271c-0316-42d0-ad6c-1d9e2b0cd6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:24.757357 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8271c-0316-42d0-ad6c-1d9e2b0cd6d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:24.758068 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8271c-0316-42d0-ad6c-1d9e2b0cd6d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:22:24.758536 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8271c-0316-42d0-ad6c-1d9e2b0cd6d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:24.760025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8271c-0316-42d0-ad6c-1d9e2b0cd6d1 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:24.772806 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1017/2019] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:22:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:22:24.777251 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c024e30-9b40-4731-adb9-d442d8c6fcc2 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:24.787536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c024e30-9b40-4731-adb9-d442d8c6fcc2 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:24.792521 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24776446-0b44-4929-b691-5b698aa899f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:24.793367 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24776446-0b44-4929-b691-5b698aa899f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:24.793792 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24776446-0b44-4929-b691-5b698aa899f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:24.794170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24776446-0b44-4929-b691-5b698aa899f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:24.795559 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24776446-0b44-4929-b691-5b698aa899f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:24.797602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24776446-0b44-4929-b691-5b698aa899f0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:24.807828 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:24.838165 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1003/2020] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:24 2022] POST /identity/v3/projects => generated 409 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:24.839675 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8271c-0316-42d0-ad6c-1d9e2b0cd6d1 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:24.853828 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8271c-0316-42d0-ad6c-1d9e2b0cd6d1 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:24.922630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814d4ee-e556-4b18-9f8f-3f5963f0b1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:24.923516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814d4ee-e556-4b18-9f8f-3f5963f0b1ca admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:24.923799 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814d4ee-e556-4b18-9f8f-3f5963f0b1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:24.924074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814d4ee-e556-4b18-9f8f-3f5963f0b1ca admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:24.924773 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814d4ee-e556-4b18-9f8f-3f5963f0b1ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:24.926683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814d4ee-e556-4b18-9f8f-3f5963f0b1ca admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:24.949603 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1018/2021] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:24 2022] POST /identity/v3/users => generated 305 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:24.954200 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24776446-0b44-4929-b691-5b698aa899f0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:24.971439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24776446-0b44-4929-b691-5b698aa899f0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:24.973616 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a3e33-9739-4efd-aa71-33e3001a06fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:24.974356 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a3e33-9739-4efd-aa71-33e3001a06fa admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:24.974703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a3e33-9739-4efd-aa71-33e3001a06fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:24.975335 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a3e33-9739-4efd-aa71-33e3001a06fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:24.976448 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a3e33-9739-4efd-aa71-33e3001a06fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:24.979303 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a3e33-9739-4efd-aa71-33e3001a06fa admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:25.003572 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1004/2022] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:25.006255 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814d4ee-e556-4b18-9f8f-3f5963f0b1ca admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:25.019777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814d4ee-e556-4b18-9f8f-3f5963f0b1ca admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:25.024887 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0fe79-c71f-4538-8e54-6fb0707416e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:25.025791 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0fe79-c71f-4538-8e54-6fb0707416e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:25.026175 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0fe79-c71f-4538-8e54-6fb0707416e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:25.026549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0fe79-c71f-4538-8e54-6fb0707416e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:25.027663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0fe79-c71f-4538-8e54-6fb0707416e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:25.029650 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0fe79-c71f-4538-8e54-6fb0707416e5 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:25.041623 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1019/2023] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:25.046515 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a3e33-9739-4efd-aa71-33e3001a06fa admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:25.058990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a3e33-9739-4efd-aa71-33e3001a06fa admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:25.065109 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b7bd2-b69d-4292-b097-f49256ad6ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:25.066125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b7bd2-b69d-4292-b097-f49256ad6ff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:25.066472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b7bd2-b69d-4292-b097-f49256ad6ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:25.066753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b7bd2-b69d-4292-b097-f49256ad6ff7 admin admin] PATH_INFO: `/v3/projects/1ac42a9df13d40b98919d72a5e65ce62/users/fa05e385ff9a4f298dec53fe60da840d/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:25.093462 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b7bd2-b69d-4292-b097-f49256ad6ff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ac42a9df13d40b98919d72a5e65ce62, user_id=fa05e385ff9a4f298dec53fe60da840d, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:25.093462 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b7bd2-b69d-4292-b097-f49256ad6ff7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:25.099388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0fe79-c71f-4538-8e54-6fb0707416e5 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:25.101434 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c8a0fe79-c71f-4538-8e54-6fb0707416e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:25.115364 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0fe79-c71f-4538-8e54-6fb0707416e5 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:25.126224 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1005/2024] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:25 2022] PUT /identity/v3/projects/1ac42a9df13d40b98919d72a5e65ce62/users/fa05e385ff9a4f298dec53fe60da840d/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:22:25.126763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:25.132729 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562acc85-d733-44de-9bdc-6bd815a20138 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:25.133091 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562acc85-d733-44de-9bdc-6bd815a20138 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:25.133379 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562acc85-d733-44de-9bdc-6bd815a20138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:25.183478 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-562acc85-d733-44de-9bdc-6bd815a20138 None None] MFA Rules not processed for user `fa05e385ff9a4f298dec53fe60da840d`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:25.186871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-562acc85-d733-44de-9bdc-6bd815a20138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:25.205130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb43f9-6997-4d2d-872c-e9134974d6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:25.206423 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb43f9-6997-4d2d-872c-e9134974d6cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:25.206862 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb43f9-6997-4d2d-872c-e9134974d6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:25.207248 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb43f9-6997-4d2d-872c-e9134974d6cc admin admin] PATH_INFO: `/v3/users/e459020eded5417584bb322d72f80884` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:25.208755 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb43f9-6997-4d2d-872c-e9134974d6cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e459020eded5417584bb322d72f80884)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:25.210832 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb43f9-6997-4d2d-872c-e9134974d6cc admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:25.288520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1006/2025] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:25 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:25.298440 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-21fb43f9-6997-4d2d-872c-e9134974d6cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e459020eded5417584bb322d72f80884'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:25.298547 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562acc85-d733-44de-9bdc-6bd815a20138 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:25.306935 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562acc85-d733-44de-9bdc-6bd815a20138 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:25.309325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-21fb43f9-6997-4d2d-872c-e9134974d6cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e459020eded5417584bb322d72f80884'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:25.310158 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-21fb43f9-6997-4d2d-872c-e9134974d6cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e459020eded5417584bb322d72f80884'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:25.311388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-21fb43f9-6997-4d2d-872c-e9134974d6cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e459020eded5417584bb322d72f80884'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:25.312804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b1a91-891c-4043-a611-a59dc121324a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:25.314107 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b1a91-891c-4043-a611-a59dc121324a admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:25.314395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b1a91-891c-4043-a611-a59dc121324a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:25.314668 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b1a91-891c-4043-a611-a59dc121324a admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:25.315487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b1a91-891c-4043-a611-a59dc121324a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:25.317306 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b1a91-891c-4043-a611-a59dc121324a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:25.320264 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-21fb43f9-6997-4d2d-872c-e9134974d6cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e459020eded5417584bb322d72f80884'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:25.335692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1020/2026] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:25 2022] DELETE /identity/v3/users/e459020eded5417584bb322d72f80884 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:25.340827 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7a73c-0070-4926-b314-5c79cb27fa98 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:25.341120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7a73c-0070-4926-b314-5c79cb27fa98 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:25.341443 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7a73c-0070-4926-b314-5c79cb27fa98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:25.350469 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1007/2027] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:25 2022] POST /identity/v3/projects => generated 409 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:25.354383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b1a91-891c-4043-a611-a59dc121324a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:25.356051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6c0b1a91-891c-4043-a611-a59dc121324a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:25.361571 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-03e7a73c-0070-4926-b314-5c79cb27fa98 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:25.362747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-03e7a73c-0070-4926-b314-5c79cb27fa98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:25.423196 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b1a91-891c-4043-a611-a59dc121324a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:25.426054 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1021/2028] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:22:25.428060 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f00ab-78fa-4c2a-a112-f63e7f00457f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:25.428769 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f00ab-78fa-4c2a-a112-f63e7f00457f admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:25.429044 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f00ab-78fa-4c2a-a112-f63e7f00457f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:25.429324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f00ab-78fa-4c2a-a112-f63e7f00457f admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:25.430023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f00ab-78fa-4c2a-a112-f63e7f00457f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:25.432006 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f00ab-78fa-4c2a-a112-f63e7f00457f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:25.436619 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e7a73c-0070-4926-b314-5c79cb27fa98 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:25.438299 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-03e7a73c-0070-4926-b314-5c79cb27fa98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:25.456888 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1008/2029] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:25 2022] POST /identity/v3/users => generated 311 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:22:25.463145 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f00ab-78fa-4c2a-a112-f63e7f00457f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:25.473866 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f00ab-78fa-4c2a-a112-f63e7f00457f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:25.479158 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669303bf-9b60-47ce-b77d-93147ee01ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:25.479916 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669303bf-9b60-47ce-b77d-93147ee01ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:25.480268 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669303bf-9b60-47ce-b77d-93147ee01ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:25.480458 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:25.480652 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669303bf-9b60-47ce-b77d-93147ee01ce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:25.481877 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669303bf-9b60-47ce-b77d-93147ee01ce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:25.483712 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669303bf-9b60-47ce-b77d-93147ee01ce3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:25.489111 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:25.499936 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1009/2030] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:22:25.501324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669303bf-9b60-47ce-b77d-93147ee01ce3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:25.509195 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e7a73c-0070-4926-b314-5c79cb27fa98 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:25.511358 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669303bf-9b60-47ce-b77d-93147ee01ce3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:25.512699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f91a3-efc3-457c-895b-55a717a7bfc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:25.513421 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f91a3-efc3-457c-895b-55a717a7bfc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:25.514210 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f91a3-efc3-457c-895b-55a717a7bfc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:25.514569 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f91a3-efc3-457c-895b-55a717a7bfc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:25.516630 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aed870-9315-443a-8d92-2f94dd4f5e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:25.517500 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aed870-9315-443a-8d92-2f94dd4f5e9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:25.517829 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aed870-9315-443a-8d92-2f94dd4f5e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:25.518055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aed870-9315-443a-8d92-2f94dd4f5e9d admin admin] PATH_INFO: `/v3/projects/5ba58c4ae7394baea46446abc1a2d7e9/users/37011663448742b091c2bc451dddafbf/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:25.523965 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f91a3-efc3-457c-895b-55a717a7bfc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:25.525396 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f91a3-efc3-457c-895b-55a717a7bfc6 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:25.533003 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aed870-9315-443a-8d92-2f94dd4f5e9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba58c4ae7394baea46446abc1a2d7e9, user_id=37011663448742b091c2bc451dddafbf, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:25.535110 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1022/2031] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:22:25.535681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aed870-9315-443a-8d92-2f94dd4f5e9d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:25.554069 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1010/2032] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:25 2022] PUT /identity/v3/projects/5ba58c4ae7394baea46446abc1a2d7e9/users/37011663448742b091c2bc451dddafbf/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:25.557967 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f91a3-efc3-457c-895b-55a717a7bfc6 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:25.569401 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f91a3-efc3-457c-895b-55a717a7bfc6 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:25.629593 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee53284-5970-4ec5-9823-af8c43b90108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:25.630349 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee53284-5970-4ec5-9823-af8c43b90108 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:25.630633 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee53284-5970-4ec5-9823-af8c43b90108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:25.630907 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee53284-5970-4ec5-9823-af8c43b90108 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:25.631748 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee53284-5970-4ec5-9823-af8c43b90108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:25.633364 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee53284-5970-4ec5-9823-af8c43b90108 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:25.647555 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1023/2033] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:25.648798 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aed870-9315-443a-8d92-2f94dd4f5e9d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:25.659831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aed870-9315-443a-8d92-2f94dd4f5e9d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:25.664380 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8925a-44b0-4a04-bb0a-1a7a2d66e1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:25.665295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8925a-44b0-4a04-bb0a-1a7a2d66e1ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:25.665574 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8925a-44b0-4a04-bb0a-1a7a2d66e1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:25.665846 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8925a-44b0-4a04-bb0a-1a7a2d66e1ff admin admin] PATH_INFO: `/v3/projects/5ba58c4ae7394baea46446abc1a2d7e9/users/37011663448742b091c2bc451dddafbf/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:25.667851 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8925a-44b0-4a04-bb0a-1a7a2d66e1ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba58c4ae7394baea46446abc1a2d7e9, user_id=37011663448742b091c2bc451dddafbf, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:25.671366 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8925a-44b0-4a04-bb0a-1a7a2d66e1ff admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:25.697869 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1011/2034] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:25 2022] PUT /identity/v3/projects/5ba58c4ae7394baea46446abc1a2d7e9/users/37011663448742b091c2bc451dddafbf/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:25.701013 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dac663-d536-4f37-a85a-390bfc5025b6 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:25.701376 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dac663-d536-4f37-a85a-390bfc5025b6 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:25.701822 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dac663-d536-4f37-a85a-390bfc5025b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:25.728745 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a9dac663-d536-4f37-a85a-390bfc5025b6 None None] MFA Rules not processed for user `37011663448742b091c2bc451dddafbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:25.730082 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a9dac663-d536-4f37-a85a-390bfc5025b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:25.818174 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1024/2035] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:25 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:25.832170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8925a-44b0-4a04-bb0a-1a7a2d66e1ff admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:25.832907 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dac663-d536-4f37-a85a-390bfc5025b6 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:25.833938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a9dac663-d536-4f37-a85a-390bfc5025b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:25.837134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dde8925a-44b0-4a04-bb0a-1a7a2d66e1ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:25.897639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:25.899947 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8925a-44b0-4a04-bb0a-1a7a2d66e1ff admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:25.903609 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2cf62-b968-47ae-aa27-c831645f2891 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:25.903860 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dac663-d536-4f37-a85a-390bfc5025b6 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:25.904996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2cf62-b968-47ae-aa27-c831645f2891 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:25.905362 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2cf62-b968-47ae-aa27-c831645f2891 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:25.905643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2cf62-b968-47ae-aa27-c831645f2891 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:25.909195 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8cd98-7859-44f7-97f5-fad3d23302d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:25.910047 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8cd98-7859-44f7-97f5-fad3d23302d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:25.910329 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8cd98-7859-44f7-97f5-fad3d23302d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:25.910607 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8cd98-7859-44f7-97f5-fad3d23302d5 admin admin] PATH_INFO: `/v3/projects/51dd501093ad4b5484ec28131c248876` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:25.911809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8cd98-7859-44f7-97f5-fad3d23302d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51dd501093ad4b5484ec28131c248876)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:25.913747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8cd98-7859-44f7-97f5-fad3d23302d5 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:25.914965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b2cf62-b968-47ae-aa27-c831645f2891 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:25.916425 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b2cf62-b968-47ae-aa27-c831645f2891 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:25.926431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1012/2036] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:25 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:25.961180 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3ad8cd98-7859-44f7-97f5-fad3d23302d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51dd501093ad4b5484ec28131c248876', 'request_id': 'req-3ad8cd98-7859-44f7-97f5-fad3d23302d5'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:25.970013 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3ad8cd98-7859-44f7-97f5-fad3d23302d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51dd501093ad4b5484ec28131c248876', 'request_id': 'req-3ad8cd98-7859-44f7-97f5-fad3d23302d5'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:25.976878 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3ad8cd98-7859-44f7-97f5-fad3d23302d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51dd501093ad4b5484ec28131c248876', 'request_id': 'req-3ad8cd98-7859-44f7-97f5-fad3d23302d5'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:25.992709 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3ad8cd98-7859-44f7-97f5-fad3d23302d5 admin admin] The token cache is being invalidate because project 51dd501093ad4b5484ec28131c248876 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:25.993084 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3ad8cd98-7859-44f7-97f5-fad3d23302d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:25.993753 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3ad8cd98-7859-44f7-97f5-fad3d23302d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:25.995405 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1025/2037] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:25 2022] DELETE /identity/v3/projects/51dd501093ad4b5484ec28131c248876 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:22:25.999965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2cf62-b968-47ae-aa27-c831645f2891 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:26.010345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2cf62-b968-47ae-aa27-c831645f2891 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:26.077169 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069de2e-5468-4c06-8a31-b5b84739437f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:26.077688 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8cd98-7859-44f7-97f5-fad3d23302d5 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:26.078048 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069de2e-5468-4c06-8a31-b5b84739437f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:26.078331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069de2e-5468-4c06-8a31-b5b84739437f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:26.078603 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069de2e-5468-4c06-8a31-b5b84739437f admin admin] PATH_INFO: `/v3/users/a8c64cb736e34d25ba0ad1ad42c30364` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:26.079173 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3ad8cd98-7859-44f7-97f5-fad3d23302d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:26.079854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1069de2e-5468-4c06-8a31-b5b84739437f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8c64cb736e34d25ba0ad1ad42c30364)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:26.081801 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1069de2e-5468-4c06-8a31-b5b84739437f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:26.131796 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1069de2e-5468-4c06-8a31-b5b84739437f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8c64cb736e34d25ba0ad1ad42c30364'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:26.132860 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1069de2e-5468-4c06-8a31-b5b84739437f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8c64cb736e34d25ba0ad1ad42c30364'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:26.133908 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1069de2e-5468-4c06-8a31-b5b84739437f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8c64cb736e34d25ba0ad1ad42c30364'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:26.139087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8cd98-7859-44f7-97f5-fad3d23302d5 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:26.144856 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5fd65-4053-425f-84b1-9119ef07a2a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:26.145369 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1069de2e-5468-4c06-8a31-b5b84739437f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8c64cb736e34d25ba0ad1ad42c30364'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:26.145506 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5fd65-4053-425f-84b1-9119ef07a2a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:26.145789 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5fd65-4053-425f-84b1-9119ef07a2a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:26.146060 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5fd65-4053-425f-84b1-9119ef07a2a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:26.147934 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d5c5fd65-4053-425f-84b1-9119ef07a2a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:26.158289 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:26.165091 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1069de2e-5468-4c06-8a31-b5b84739437f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8c64cb736e34d25ba0ad1ad42c30364'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:26.183658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1013/2038] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:25 2022] DELETE /identity/v3/users/a8c64cb736e34d25ba0ad1ad42c30364 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:26.213013 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5fd65-4053-425f-84b1-9119ef07a2a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:26.214445 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5fd65-4053-425f-84b1-9119ef07a2a6 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:26.225070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1026/2039] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:26 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:22:26.575351 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069de2e-5468-4c06-8a31-b5b84739437f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:26.575351 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1069de2e-5468-4c06-8a31-b5b84739437f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:26.663386 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069de2e-5468-4c06-8a31-b5b84739437f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:26.668306 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c79204-ef2e-434e-b860-f1d10c848972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:26.669254 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c79204-ef2e-434e-b860-f1d10c848972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:26.670112 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c79204-ef2e-434e-b860-f1d10c848972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:26.670572 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c79204-ef2e-434e-b860-f1d10c848972 admin admin] PATH_INFO: `/v3/projects/044a09a891044f21a58b8979c2bf5a40` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:26.671859 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c79204-ef2e-434e-b860-f1d10c848972 admin admin] RBAC: Authorizing `identity:delete_project(project_id=044a09a891044f21a58b8979c2bf5a40)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:26.673850 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c79204-ef2e-434e-b860-f1d10c848972 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:26.718721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d4c79204-ef2e-434e-b860-f1d10c848972 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '044a09a891044f21a58b8979c2bf5a40', 'request_id': 'req-d4c79204-ef2e-434e-b860-f1d10c848972'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:26.726275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d4c79204-ef2e-434e-b860-f1d10c848972 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '044a09a891044f21a58b8979c2bf5a40', 'request_id': 'req-d4c79204-ef2e-434e-b860-f1d10c848972'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:26.731609 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d4c79204-ef2e-434e-b860-f1d10c848972 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '044a09a891044f21a58b8979c2bf5a40', 'request_id': 'req-d4c79204-ef2e-434e-b860-f1d10c848972'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:26.742870 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d4c79204-ef2e-434e-b860-f1d10c848972 admin admin] The token cache is being invalidate because project 044a09a891044f21a58b8979c2bf5a40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:26.743157 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d4c79204-ef2e-434e-b860-f1d10c848972 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:26.743804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d4c79204-ef2e-434e-b860-f1d10c848972 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:26.745349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1014/2040] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:26 2022] DELETE /identity/v3/projects/044a09a891044f21a58b8979c2bf5a40 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:26.750919 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aae920-e751-420b-8601-b8fd7e39552f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:26.751414 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aae920-e751-420b-8601-b8fd7e39552f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:26.751505 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aae920-e751-420b-8601-b8fd7e39552f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:26.769217 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-93aae920-e751-420b-8601-b8fd7e39552f None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:26.770596 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-93aae920-e751-420b-8601-b8fd7e39552f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:26.803676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:26.830913 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1027/2041] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:26.839052 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec20b61-4190-403e-bcc6-2649111ed27d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:26.839430 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec20b61-4190-403e-bcc6-2649111ed27d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:26.839771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec20b61-4190-403e-bcc6-2649111ed27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:26.860095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-fec20b61-4190-403e-bcc6-2649111ed27d None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:26.861422 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fec20b61-4190-403e-bcc6-2649111ed27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:26.869649 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1015/2042] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:26.874339 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aae920-e751-420b-8601-b8fd7e39552f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:26.884720 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aae920-e751-420b-8601-b8fd7e39552f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:26.889717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af6d54-ed6d-4e7d-b667-35928eb1a7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:26.890379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af6d54-ed6d-4e7d-b667-35928eb1a7b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:26.890664 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af6d54-ed6d-4e7d-b667-35928eb1a7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:26.891090 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af6d54-ed6d-4e7d-b667-35928eb1a7b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:26.891874 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af6d54-ed6d-4e7d-b667-35928eb1a7b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:22:26.892168 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af6d54-ed6d-4e7d-b667-35928eb1a7b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:26.893764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af6d54-ed6d-4e7d-b667-35928eb1a7b4 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:26.906251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1028/2043] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:22:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:22:26.910732 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec20b61-4190-403e-bcc6-2649111ed27d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:26.920667 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec20b61-4190-403e-bcc6-2649111ed27d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:26.925439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e5099-134f-488a-b3bd-61fff71a8148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:26.926084 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e5099-134f-488a-b3bd-61fff71a8148 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:26.926364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e5099-134f-488a-b3bd-61fff71a8148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:26.926636 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e5099-134f-488a-b3bd-61fff71a8148 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:26.927407 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e5099-134f-488a-b3bd-61fff71a8148 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:26.929276 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e5099-134f-488a-b3bd-61fff71a8148 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:26.954442 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1016/2044] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:26 2022] POST /identity/v3/projects => generated 365 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:26.958017 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af6d54-ed6d-4e7d-b667-35928eb1a7b4 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:26.968715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af6d54-ed6d-4e7d-b667-35928eb1a7b4 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:27.025777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457d9f7-e191-48e7-a435-38795f55938b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:27.026439 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457d9f7-e191-48e7-a435-38795f55938b admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:27.026747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457d9f7-e191-48e7-a435-38795f55938b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:27.027022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457d9f7-e191-48e7-a435-38795f55938b admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:27.027743 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457d9f7-e191-48e7-a435-38795f55938b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:27.030181 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457d9f7-e191-48e7-a435-38795f55938b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:27.050562 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1029/2045] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:26 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:27.054887 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e5099-134f-488a-b3bd-61fff71a8148 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:27.065919 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e5099-134f-488a-b3bd-61fff71a8148 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:27.070683 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df103b-233c-4c7e-aff6-a7f2ccddac6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:27.071499 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df103b-233c-4c7e-aff6-a7f2ccddac6f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:27.071783 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df103b-233c-4c7e-aff6-a7f2ccddac6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:27.072102 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df103b-233c-4c7e-aff6-a7f2ccddac6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:27.072827 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df103b-233c-4c7e-aff6-a7f2ccddac6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:27.074413 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df103b-233c-4c7e-aff6-a7f2ccddac6f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:27.085632 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1017/2046] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:27.089563 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457d9f7-e191-48e7-a435-38795f55938b admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:27.099591 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457d9f7-e191-48e7-a435-38795f55938b admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:27.104395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9c4d9-2b90-46ce-a562-81b3e8316e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:27.105052 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9c4d9-2b90-46ce-a562-81b3e8316e3c admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:27.105337 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9c4d9-2b90-46ce-a562-81b3e8316e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:27.105615 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9c4d9-2b90-46ce-a562-81b3e8316e3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:27.106347 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b9c4d9-2b90-46ce-a562-81b3e8316e3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:27.108616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b9c4d9-2b90-46ce-a562-81b3e8316e3c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:27.119154 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1030/2047] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:27.124778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df103b-233c-4c7e-aff6-a7f2ccddac6f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:27.139118 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df103b-233c-4c7e-aff6-a7f2ccddac6f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:27.143807 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213a732-fcd5-4b09-9c1c-31c73c3d6982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:27.144685 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213a732-fcd5-4b09-9c1c-31c73c3d6982 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:27.144996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213a732-fcd5-4b09-9c1c-31c73c3d6982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:27.145271 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213a732-fcd5-4b09-9c1c-31c73c3d6982 admin admin] PATH_INFO: `/v3/projects/4b6b0590e8ce4760a06e634b58beb312/users/2385dea59da944b693007bee2a880433/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:27.160683 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213a732-fcd5-4b09-9c1c-31c73c3d6982 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b6b0590e8ce4760a06e634b58beb312, user_id=2385dea59da944b693007bee2a880433, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:27.163736 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213a732-fcd5-4b09-9c1c-31c73c3d6982 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:27.177386 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1018/2048] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:27 2022] PUT /identity/v3/projects/4b6b0590e8ce4760a06e634b58beb312/users/2385dea59da944b693007bee2a880433/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:27.188199 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00162c8-030a-416b-b957-f66f802a639a None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:27.188521 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00162c8-030a-416b-b957-f66f802a639a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:27.188802 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00162c8-030a-416b-b957-f66f802a639a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:27.215438 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-c00162c8-030a-416b-b957-f66f802a639a None None] MFA Rules not processed for user `2385dea59da944b693007bee2a880433`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:27.216652 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c00162c8-030a-416b-b957-f66f802a639a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:27.279242 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1031/2049] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:27 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:27.283639 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213a732-fcd5-4b09-9c1c-31c73c3d6982 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:27.296623 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:27.347986 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213a732-fcd5-4b09-9c1c-31c73c3d6982 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:27.352817 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf855be4-dbac-4719-bac3-0d84df64bb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:27.353475 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf855be4-dbac-4719-bac3-0d84df64bb92 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:27.353761 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf855be4-dbac-4719-bac3-0d84df64bb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:27.354039 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf855be4-dbac-4719-bac3-0d84df64bb92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:27.354727 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf855be4-dbac-4719-bac3-0d84df64bb92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:27.356674 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf855be4-dbac-4719-bac3-0d84df64bb92 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:27.383968 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1019/2050] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:27 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:27.388148 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00162c8-030a-416b-b957-f66f802a639a None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:27.399953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00162c8-030a-416b-b957-f66f802a639a None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:27.435039 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:27.460704 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4a9d7-de42-428b-8c9d-0b327f0fa51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:27.461384 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4a9d7-de42-428b-8c9d-0b327f0fa51c admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:27.461677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4a9d7-de42-428b-8c9d-0b327f0fa51c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:27.461954 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4a9d7-de42-428b-8c9d-0b327f0fa51c admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:27.462645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd4a9d7-de42-428b-8c9d-0b327f0fa51c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:27.464602 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd4a9d7-de42-428b-8c9d-0b327f0fa51c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:27.485092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1032/2051] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:27 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:22:27.489198 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf855be4-dbac-4719-bac3-0d84df64bb92 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:27.500441 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf855be4-dbac-4719-bac3-0d84df64bb92 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:27.505050 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0148314-d1e9-45f9-a013-5147b9a944e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:27.505707 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0148314-d1e9-45f9-a013-5147b9a944e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:27.505990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0148314-d1e9-45f9-a013-5147b9a944e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:27.506263 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0148314-d1e9-45f9-a013-5147b9a944e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:27.506986 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0148314-d1e9-45f9-a013-5147b9a944e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:27.508599 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0148314-d1e9-45f9-a013-5147b9a944e7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:27.519371 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1020/2052] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:27.523431 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4a9d7-de42-428b-8c9d-0b327f0fa51c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:27.533402 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4a9d7-de42-428b-8c9d-0b327f0fa51c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:27.538046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a272cef5-c8f9-479e-b8e0-817f0d41bd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:27.538934 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272cef5-c8f9-479e-b8e0-817f0d41bd4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:27.539276 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272cef5-c8f9-479e-b8e0-817f0d41bd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:27.539558 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272cef5-c8f9-479e-b8e0-817f0d41bd4e admin admin] PATH_INFO: `/v3/projects/c4bd00a4cfd94739b23da26e08c24ec8/users/6c01b5f3060341329afd16bb093dc8de/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:27.554729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a272cef5-c8f9-479e-b8e0-817f0d41bd4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4bd00a4cfd94739b23da26e08c24ec8, user_id=6c01b5f3060341329afd16bb093dc8de, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:27.557398 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a272cef5-c8f9-479e-b8e0-817f0d41bd4e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:27.581049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1033/2053] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:27 2022] PUT /identity/v3/projects/c4bd00a4cfd94739b23da26e08c24ec8/users/6c01b5f3060341329afd16bb093dc8de/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:27.584927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0148314-d1e9-45f9-a013-5147b9a944e7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:27.595546 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0148314-d1e9-45f9-a013-5147b9a944e7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:27.655492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9b66c-9c7b-452b-acae-a1fff2680a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:27.656172 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9b66c-9c7b-452b-acae-a1fff2680a00 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:27.656454 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9b66c-9c7b-452b-acae-a1fff2680a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:27.656728 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9b66c-9c7b-452b-acae-a1fff2680a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:27.657458 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9b66c-9c7b-452b-acae-a1fff2680a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:27.659057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9b66c-9c7b-452b-acae-a1fff2680a00 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:27.670795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1021/2054] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:27.674460 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a272cef5-c8f9-479e-b8e0-817f0d41bd4e admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:27.684624 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a272cef5-c8f9-479e-b8e0-817f0d41bd4e admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:27.692169 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c720d6b6-078d-44ed-b05f-a5050a0d73c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:27.693060 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720d6b6-078d-44ed-b05f-a5050a0d73c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:27.693339 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720d6b6-078d-44ed-b05f-a5050a0d73c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:27.693616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720d6b6-078d-44ed-b05f-a5050a0d73c7 admin admin] PATH_INFO: `/v3/projects/c4bd00a4cfd94739b23da26e08c24ec8/users/6c01b5f3060341329afd16bb093dc8de/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:27.695604 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c720d6b6-078d-44ed-b05f-a5050a0d73c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4bd00a4cfd94739b23da26e08c24ec8, user_id=6c01b5f3060341329afd16bb093dc8de, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:27.698230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c720d6b6-078d-44ed-b05f-a5050a0d73c7 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:27.709249 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1034/2055] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:27 2022] PUT /identity/v3/projects/c4bd00a4cfd94739b23da26e08c24ec8/users/6c01b5f3060341329afd16bb093dc8de/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:27.721280 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c201916-846a-4d5e-8c2e-f29307a2822a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:27.721800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c201916-846a-4d5e-8c2e-f29307a2822a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:27.721968 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c201916-846a-4d5e-8c2e-f29307a2822a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:27.749775 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0c201916-846a-4d5e-8c2e-f29307a2822a None None] MFA Rules not processed for user `6c01b5f3060341329afd16bb093dc8de`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:27.751127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0c201916-846a-4d5e-8c2e-f29307a2822a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:27.818226 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1022/2056] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:27 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:27.829393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c720d6b6-078d-44ed-b05f-a5050a0d73c7 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:27.831100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c720d6b6-078d-44ed-b05f-a5050a0d73c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:27.890325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c720d6b6-078d-44ed-b05f-a5050a0d73c7 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:27.893987 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d63bb-02ce-419f-9f1e-044bbb015946 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:27.894626 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d63bb-02ce-419f-9f1e-044bbb015946 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:27.894906 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d63bb-02ce-419f-9f1e-044bbb015946 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:27.895179 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d63bb-02ce-419f-9f1e-044bbb015946 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:27.904716 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d63bb-02ce-419f-9f1e-044bbb015946 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:27.906157 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d63bb-02ce-419f-9f1e-044bbb015946 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:27.915840 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1035/2057] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:27 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:27.990922 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c201916-846a-4d5e-8c2e-f29307a2822a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:28.000950 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c201916-846a-4d5e-8c2e-f29307a2822a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:28.004517 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49833b2-cc1a-4866-a34c-c9b658e67b21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:28.005161 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49833b2-cc1a-4866-a34c-c9b658e67b21 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:28.005443 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49833b2-cc1a-4866-a34c-c9b658e67b21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:28.005717 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49833b2-cc1a-4866-a34c-c9b658e67b21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:28.009810 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:28.063932 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49833b2-cc1a-4866-a34c-c9b658e67b21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:28.065344 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49833b2-cc1a-4866-a34c-c9b658e67b21 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:28.074406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1023/2058] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:27 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:30.978420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d63bb-02ce-419f-9f1e-044bbb015946 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:30.980065 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-115d63bb-02ce-419f-9f1e-044bbb015946 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:31.048947 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d63bb-02ce-419f-9f1e-044bbb015946 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:31.054233 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd78afb-be2b-447a-a9a1-b1ad56d3ae7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:31.055154 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd78afb-be2b-447a-a9a1-b1ad56d3ae7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:31.055517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd78afb-be2b-447a-a9a1-b1ad56d3ae7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:31.055821 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd78afb-be2b-447a-a9a1-b1ad56d3ae7d admin admin] PATH_INFO: `/v3/users/ec9030707c8f4b3cba79a96901cac687` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:31.057114 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd78afb-be2b-447a-a9a1-b1ad56d3ae7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec9030707c8f4b3cba79a96901cac687)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:31.059078 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd78afb-be2b-447a-a9a1-b1ad56d3ae7d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:31.105548 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:31.127381 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7dd78afb-be2b-447a-a9a1-b1ad56d3ae7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec9030707c8f4b3cba79a96901cac687'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:31.144787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7dd78afb-be2b-447a-a9a1-b1ad56d3ae7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec9030707c8f4b3cba79a96901cac687'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:31.147406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7dd78afb-be2b-447a-a9a1-b1ad56d3ae7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec9030707c8f4b3cba79a96901cac687'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:31.148149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7dd78afb-be2b-447a-a9a1-b1ad56d3ae7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec9030707c8f4b3cba79a96901cac687'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:31.176190 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7dd78afb-be2b-447a-a9a1-b1ad56d3ae7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec9030707c8f4b3cba79a96901cac687'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:31.194908 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1036/2059] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:30 2022] DELETE /identity/v3/users/ec9030707c8f4b3cba79a96901cac687 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:31.199271 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02198dea-0ba3-4056-bb07-1930bc50a081 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:31.200168 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02198dea-0ba3-4056-bb07-1930bc50a081 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:31.201162 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02198dea-0ba3-4056-bb07-1930bc50a081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:31.221464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-02198dea-0ba3-4056-bb07-1930bc50a081 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:31.222931 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-02198dea-0ba3-4056-bb07-1930bc50a081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:31.301930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1024/2060] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:31.313449 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd78afb-be2b-447a-a9a1-b1ad56d3ae7d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:31.314879 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7dd78afb-be2b-447a-a9a1-b1ad56d3ae7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:31.389294 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd78afb-be2b-447a-a9a1-b1ad56d3ae7d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:31.393212 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b11ca-0ab8-44a6-9bca-3a495b446b80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:31.393938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b11ca-0ab8-44a6-9bca-3a495b446b80 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:31.394221 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b11ca-0ab8-44a6-9bca-3a495b446b80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:31.394497 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b11ca-0ab8-44a6-9bca-3a495b446b80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:31.405388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b11ca-0ab8-44a6-9bca-3a495b446b80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:31.407150 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b11ca-0ab8-44a6-9bca-3a495b446b80 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:31.422846 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1037/2061] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:31.708460 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02198dea-0ba3-4056-bb07-1930bc50a081 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:31.710035 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-02198dea-0ba3-4056-bb07-1930bc50a081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:31.722144 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02198dea-0ba3-4056-bb07-1930bc50a081 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:31.727013 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e965ca-a396-452f-93c4-36725de78ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:31.727913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e965ca-a396-452f-93c4-36725de78ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:31.728191 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e965ca-a396-452f-93c4-36725de78ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:31.728467 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e965ca-a396-452f-93c4-36725de78ca0 admin admin] PATH_INFO: `/v3/projects/438a1d9ddc1748229a377a64cd453792` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:31.729729 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e965ca-a396-452f-93c4-36725de78ca0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=438a1d9ddc1748229a377a64cd453792)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:31.731671 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e965ca-a396-452f-93c4-36725de78ca0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:31.784950 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b9e965ca-a396-452f-93c4-36725de78ca0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '438a1d9ddc1748229a377a64cd453792', 'request_id': 'req-b9e965ca-a396-452f-93c4-36725de78ca0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:31.792065 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b9e965ca-a396-452f-93c4-36725de78ca0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '438a1d9ddc1748229a377a64cd453792', 'request_id': 'req-b9e965ca-a396-452f-93c4-36725de78ca0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:31.798997 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b9e965ca-a396-452f-93c4-36725de78ca0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '438a1d9ddc1748229a377a64cd453792', 'request_id': 'req-b9e965ca-a396-452f-93c4-36725de78ca0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:31.809849 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b9e965ca-a396-452f-93c4-36725de78ca0 admin admin] The token cache is being invalidate because project 438a1d9ddc1748229a377a64cd453792 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:31.810755 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b9e965ca-a396-452f-93c4-36725de78ca0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:31.811581 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b9e965ca-a396-452f-93c4-36725de78ca0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:31.813144 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1025/2062] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:31 2022] DELETE /identity/v3/projects/438a1d9ddc1748229a377a64cd453792 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:31.817197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b11ca-0ab8-44a6-9bca-3a495b446b80 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:31.826962 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b11ca-0ab8-44a6-9bca-3a495b446b80 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:31.882250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ce627-cf6e-4d6d-b172-bcbada26f9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:31.883150 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ce627-cf6e-4d6d-b172-bcbada26f9c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:31.883540 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ce627-cf6e-4d6d-b172-bcbada26f9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:31.883812 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ce627-cf6e-4d6d-b172-bcbada26f9c2 admin admin] PATH_INFO: `/v3/users/a84ef3cce52e46c7a56f70f234f788e9` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:31.885316 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ce627-cf6e-4d6d-b172-bcbada26f9c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a84ef3cce52e46c7a56f70f234f788e9)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:31.887314 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ce627-cf6e-4d6d-b172-bcbada26f9c2 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:31.933606 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d54ce627-cf6e-4d6d-b172-bcbada26f9c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a84ef3cce52e46c7a56f70f234f788e9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:31.939756 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:31.948612 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d54ce627-cf6e-4d6d-b172-bcbada26f9c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a84ef3cce52e46c7a56f70f234f788e9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:31.949682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d54ce627-cf6e-4d6d-b172-bcbada26f9c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a84ef3cce52e46c7a56f70f234f788e9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:31.950331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d54ce627-cf6e-4d6d-b172-bcbada26f9c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a84ef3cce52e46c7a56f70f234f788e9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:31.957645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d54ce627-cf6e-4d6d-b172-bcbada26f9c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a84ef3cce52e46c7a56f70f234f788e9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:31.970613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1038/2063] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:31 2022] DELETE /identity/v3/users/a84ef3cce52e46c7a56f70f234f788e9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:32.247982 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e965ca-a396-452f-93c4-36725de78ca0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:32.249626 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b9e965ca-a396-452f-93c4-36725de78ca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:32.265176 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e965ca-a396-452f-93c4-36725de78ca0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:32.292510 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:32.327302 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a4b7f-b35a-46f0-a7ea-64a2d66ab762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:32.328486 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a4b7f-b35a-46f0-a7ea-64a2d66ab762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:32.328772 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a4b7f-b35a-46f0-a7ea-64a2d66ab762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:32.329048 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a4b7f-b35a-46f0-a7ea-64a2d66ab762 admin admin] PATH_INFO: `/v3/projects/a0f47622605542f090807f8c2aec0cbe` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:32.335056 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a4b7f-b35a-46f0-a7ea-64a2d66ab762 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0f47622605542f090807f8c2aec0cbe)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:32.335056 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a4b7f-b35a-46f0-a7ea-64a2d66ab762 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:32.383257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a93a4b7f-b35a-46f0-a7ea-64a2d66ab762 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0f47622605542f090807f8c2aec0cbe', 'request_id': 'req-a93a4b7f-b35a-46f0-a7ea-64a2d66ab762'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:32.393982 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a93a4b7f-b35a-46f0-a7ea-64a2d66ab762 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0f47622605542f090807f8c2aec0cbe', 'request_id': 'req-a93a4b7f-b35a-46f0-a7ea-64a2d66ab762'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:32.404046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a93a4b7f-b35a-46f0-a7ea-64a2d66ab762 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0f47622605542f090807f8c2aec0cbe', 'request_id': 'req-a93a4b7f-b35a-46f0-a7ea-64a2d66ab762'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:32.419928 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a93a4b7f-b35a-46f0-a7ea-64a2d66ab762 admin admin] The token cache is being invalidate because project a0f47622605542f090807f8c2aec0cbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:32.420926 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a93a4b7f-b35a-46f0-a7ea-64a2d66ab762 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:32.421847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a93a4b7f-b35a-46f0-a7ea-64a2d66ab762 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:32.426405 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1026/2064] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:32 2022] DELETE /identity/v3/projects/a0f47622605542f090807f8c2aec0cbe => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:32.427734 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ce627-cf6e-4d6d-b172-bcbada26f9c2 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:32.434416 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d54ce627-cf6e-4d6d-b172-bcbada26f9c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:32.450349 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ce627-cf6e-4d6d-b172-bcbada26f9c2 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:32.511850 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40b709-6b2a-4ad1-9879-8719bf2230ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:32.512729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40b709-6b2a-4ad1-9879-8719bf2230ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:32.513608 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40b709-6b2a-4ad1-9879-8719bf2230ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:32.514073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40b709-6b2a-4ad1-9879-8719bf2230ac admin admin] PATH_INFO: `/v3/users/97aefb29de8a429aa0c06a843825ab52` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:32.515471 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40b709-6b2a-4ad1-9879-8719bf2230ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=97aefb29de8a429aa0c06a843825ab52)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:32.517394 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40b709-6b2a-4ad1-9879-8719bf2230ac admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:32.570971 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-8a40b709-6b2a-4ad1-9879-8719bf2230ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97aefb29de8a429aa0c06a843825ab52'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:32.584418 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-8a40b709-6b2a-4ad1-9879-8719bf2230ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97aefb29de8a429aa0c06a843825ab52'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:32.585203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-8a40b709-6b2a-4ad1-9879-8719bf2230ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97aefb29de8a429aa0c06a843825ab52'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:32.585974 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-8a40b709-6b2a-4ad1-9879-8719bf2230ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97aefb29de8a429aa0c06a843825ab52'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:32.593199 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-8a40b709-6b2a-4ad1-9879-8719bf2230ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97aefb29de8a429aa0c06a843825ab52'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:32.607638 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1039/2065] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:32 2022] DELETE /identity/v3/users/97aefb29de8a429aa0c06a843825ab52 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:32.969274 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a4b7f-b35a-46f0-a7ea-64a2d66ab762 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:32.970962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a93a4b7f-b35a-46f0-a7ea-64a2d66ab762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:32.993233 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a4b7f-b35a-46f0-a7ea-64a2d66ab762 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:33.053638 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef84b56-f29d-4609-b5b4-22a73f506fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:33.054502 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef84b56-f29d-4609-b5b4-22a73f506fe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:33.054783 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef84b56-f29d-4609-b5b4-22a73f506fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:33.055058 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef84b56-f29d-4609-b5b4-22a73f506fe4 admin admin] PATH_INFO: `/v3/projects/9f7031aacc50426cbdedeebcf6016e8c` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:33.056317 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef84b56-f29d-4609-b5b4-22a73f506fe4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f7031aacc50426cbdedeebcf6016e8c)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:33.058284 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef84b56-f29d-4609-b5b4-22a73f506fe4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:33.105228 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7ef84b56-f29d-4609-b5b4-22a73f506fe4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f7031aacc50426cbdedeebcf6016e8c', 'request_id': 'req-7ef84b56-f29d-4609-b5b4-22a73f506fe4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:33.113840 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7ef84b56-f29d-4609-b5b4-22a73f506fe4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f7031aacc50426cbdedeebcf6016e8c', 'request_id': 'req-7ef84b56-f29d-4609-b5b4-22a73f506fe4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:33.117752 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:33.120099 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7ef84b56-f29d-4609-b5b4-22a73f506fe4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f7031aacc50426cbdedeebcf6016e8c', 'request_id': 'req-7ef84b56-f29d-4609-b5b4-22a73f506fe4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:33.132806 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7ef84b56-f29d-4609-b5b4-22a73f506fe4 admin admin] The token cache is being invalidate because project 9f7031aacc50426cbdedeebcf6016e8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:33.133099 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7ef84b56-f29d-4609-b5b4-22a73f506fe4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:33.133734 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7ef84b56-f29d-4609-b5b4-22a73f506fe4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:33.135302 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1027/2066] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:32 2022] DELETE /identity/v3/projects/9f7031aacc50426cbdedeebcf6016e8c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:33.141693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fbab8-6098-48fb-a7fb-ab56d368e8d3 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:33.141980 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fbab8-6098-48fb-a7fb-ab56d368e8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:33.142257 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fbab8-6098-48fb-a7fb-ab56d368e8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:33.162864 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-df8fbab8-6098-48fb-a7fb-ab56d368e8d3 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:33.164136 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-df8fbab8-6098-48fb-a7fb-ab56d368e8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:33.220150 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:33.236142 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1040/2067] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:33.244092 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127b8de-05c9-4cbc-9b5c-2b5d8b1b0159 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:33.244382 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127b8de-05c9-4cbc-9b5c-2b5d8b1b0159 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:33.244661 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127b8de-05c9-4cbc-9b5c-2b5d8b1b0159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:33.264971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6127b8de-05c9-4cbc-9b5c-2b5d8b1b0159 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:33.266176 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6127b8de-05c9-4cbc-9b5c-2b5d8b1b0159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:33.274178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1028/2068] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:33.278489 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8fbab8-6098-48fb-a7fb-ab56d368e8d3 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:33.289673 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8fbab8-6098-48fb-a7fb-ab56d368e8d3 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:33.294869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d794a2f-6e0d-491b-b004-d625c92d9169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:33.295938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d794a2f-6e0d-491b-b004-d625c92d9169 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:33.296258 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d794a2f-6e0d-491b-b004-d625c92d9169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:33.296532 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d794a2f-6e0d-491b-b004-d625c92d9169 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:33.297262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d794a2f-6e0d-491b-b004-d625c92d9169 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:22:33.297565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d794a2f-6e0d-491b-b004-d625c92d9169 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:33.299196 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d794a2f-6e0d-491b-b004-d625c92d9169 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:33.311871 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1041/2069] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:22:33.316405 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127b8de-05c9-4cbc-9b5c-2b5d8b1b0159 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:33.325399 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127b8de-05c9-4cbc-9b5c-2b5d8b1b0159 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:33.330345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee13ab9-aa85-46ec-94c7-3c34c74e7d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:33.330987 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee13ab9-aa85-46ec-94c7-3c34c74e7d73 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:33.331294 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee13ab9-aa85-46ec-94c7-3c34c74e7d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:33.331568 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee13ab9-aa85-46ec-94c7-3c34c74e7d73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:33.332245 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee13ab9-aa85-46ec-94c7-3c34c74e7d73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:33.334121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee13ab9-aa85-46ec-94c7-3c34c74e7d73 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:33.359574 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1029/2070] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:33 2022] POST /identity/v3/projects => generated 393 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:22:33.363691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d794a2f-6e0d-491b-b004-d625c92d9169 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:33.374258 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d794a2f-6e0d-491b-b004-d625c92d9169 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:33.432618 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e49ac-1948-4f40-be90-f2df1c2ac9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:33.433316 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e49ac-1948-4f40-be90-f2df1c2ac9a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:33.433558 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e49ac-1948-4f40-be90-f2df1c2ac9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:33.433837 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e49ac-1948-4f40-be90-f2df1c2ac9a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:33.434527 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e49ac-1948-4f40-be90-f2df1c2ac9a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:33.437140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e49ac-1948-4f40-be90-f2df1c2ac9a0 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:33.457863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1042/2071] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:33 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:33.461798 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee13ab9-aa85-46ec-94c7-3c34c74e7d73 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:33.472604 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee13ab9-aa85-46ec-94c7-3c34c74e7d73 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:33.477696 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea8dc0-a739-4791-9169-302a924c67ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:33.478369 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea8dc0-a739-4791-9169-302a924c67ce admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:33.478676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea8dc0-a739-4791-9169-302a924c67ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:33.478931 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea8dc0-a739-4791-9169-302a924c67ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:33.479713 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea8dc0-a739-4791-9169-302a924c67ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:33.481299 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea8dc0-a739-4791-9169-302a924c67ce admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:33.495605 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1030/2072] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:22:33.500408 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e49ac-1948-4f40-be90-f2df1c2ac9a0 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:33.510577 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e49ac-1948-4f40-be90-f2df1c2ac9a0 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:33.515313 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78499b7a-8f7a-4184-97d4-77d36faf5a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:33.515974 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78499b7a-8f7a-4184-97d4-77d36faf5a7c admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:33.516260 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78499b7a-8f7a-4184-97d4-77d36faf5a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:33.516537 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78499b7a-8f7a-4184-97d4-77d36faf5a7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:33.517295 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78499b7a-8f7a-4184-97d4-77d36faf5a7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:33.518884 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78499b7a-8f7a-4184-97d4-77d36faf5a7c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:33.531172 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1043/2073] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:33.535994 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea8dc0-a739-4791-9169-302a924c67ce admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:33.547562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea8dc0-a739-4791-9169-302a924c67ce admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:33.552250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d71b2-b522-4ce9-acee-a239edeeecd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:33.553134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d71b2-b522-4ce9-acee-a239edeeecd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:33.553443 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d71b2-b522-4ce9-acee-a239edeeecd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:33.553721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d71b2-b522-4ce9-acee-a239edeeecd6 admin admin] PATH_INFO: `/v3/projects/1f61c91a015645b0a4722285025a67d7/users/88f98c5b01d548b1b62f1ff903f4b7bf/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:33.570317 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d71b2-b522-4ce9-acee-a239edeeecd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f61c91a015645b0a4722285025a67d7, user_id=88f98c5b01d548b1b62f1ff903f4b7bf, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:33.572928 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d71b2-b522-4ce9-acee-a239edeeecd6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:33.588150 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1031/2074] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:33 2022] PUT /identity/v3/projects/1f61c91a015645b0a4722285025a67d7/users/88f98c5b01d548b1b62f1ff903f4b7bf/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:33.600638 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e477a-fdf9-441f-b388-26bd22d8c5d9 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:33.600925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e477a-fdf9-441f-b388-26bd22d8c5d9 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:33.601202 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e477a-fdf9-441f-b388-26bd22d8c5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:33.628798 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-bb0e477a-fdf9-441f-b388-26bd22d8c5d9 None None] MFA Rules not processed for user `88f98c5b01d548b1b62f1ff903f4b7bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:33.630000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-bb0e477a-fdf9-441f-b388-26bd22d8c5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:33.705221 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1044/2075] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:33.709697 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d71b2-b522-4ce9-acee-a239edeeecd6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:33.772143 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d71b2-b522-4ce9-acee-a239edeeecd6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:33.776711 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655af2d-7d68-4f20-92e4-d18be7d7d4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:33.777369 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655af2d-7d68-4f20-92e4-d18be7d7d4b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:33.777653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655af2d-7d68-4f20-92e4-d18be7d7d4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:33.777934 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655af2d-7d68-4f20-92e4-d18be7d7d4b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:33.778667 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5655af2d-7d68-4f20-92e4-d18be7d7d4b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:33.780731 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5655af2d-7d68-4f20-92e4-d18be7d7d4b5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:33.805080 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1032/2076] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:33 2022] POST /identity/v3/projects => generated 393 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:33.809158 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e477a-fdf9-441f-b388-26bd22d8c5d9 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:33.820911 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e477a-fdf9-441f-b388-26bd22d8c5d9 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:33.874476 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:33.892118 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2bc99-5e13-4aaf-8d8c-37749d33d744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:33.892460 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2bc99-5e13-4aaf-8d8c-37749d33d744 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:33.892748 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2bc99-5e13-4aaf-8d8c-37749d33d744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:33.893026 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2bc99-5e13-4aaf-8d8c-37749d33d744 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:33.893790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed2bc99-5e13-4aaf-8d8c-37749d33d744 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:33.895946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed2bc99-5e13-4aaf-8d8c-37749d33d744 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:33.921450 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1045/2077] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:33 2022] POST /identity/v3/users => generated 303 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:33.923048 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655af2d-7d68-4f20-92e4-d18be7d7d4b5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:33.933982 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655af2d-7d68-4f20-92e4-d18be7d7d4b5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:33.939583 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8bbbc-a138-4d95-8e12-d6f7ce79952c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:33.940283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8bbbc-a138-4d95-8e12-d6f7ce79952c admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:33.940568 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8bbbc-a138-4d95-8e12-d6f7ce79952c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:33.940845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8bbbc-a138-4d95-8e12-d6f7ce79952c admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:33.941581 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8bbbc-a138-4d95-8e12-d6f7ce79952c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:33.943170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8bbbc-a138-4d95-8e12-d6f7ce79952c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:33.954521 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1033/2078] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:33.958904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2bc99-5e13-4aaf-8d8c-37749d33d744 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:33.970695 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2bc99-5e13-4aaf-8d8c-37749d33d744 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:33.976084 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3814f-06e5-44ab-8237-e25ab9b5daf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:33.977011 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3814f-06e5-44ab-8237-e25ab9b5daf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:33.977298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3814f-06e5-44ab-8237-e25ab9b5daf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:33.977580 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3814f-06e5-44ab-8237-e25ab9b5daf5 admin admin] PATH_INFO: `/v3/projects/7b3a6c1570e64ea786354a7f37a4acea/users/d93bfad823684f438dd7ef6078219547/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:34.006060 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3814f-06e5-44ab-8237-e25ab9b5daf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b3a6c1570e64ea786354a7f37a4acea, user_id=d93bfad823684f438dd7ef6078219547, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:34.008761 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3814f-06e5-44ab-8237-e25ab9b5daf5 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:34.024668 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1046/2079] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:33 2022] PUT /identity/v3/projects/7b3a6c1570e64ea786354a7f37a4acea/users/d93bfad823684f438dd7ef6078219547/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:34.028689 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8bbbc-a138-4d95-8e12-d6f7ce79952c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:34.038618 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8bbbc-a138-4d95-8e12-d6f7ce79952c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:34.094315 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610957c3-da18-44a6-afb9-0b3c5cdb3799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:34.094985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610957c3-da18-44a6-afb9-0b3c5cdb3799 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:34.095306 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610957c3-da18-44a6-afb9-0b3c5cdb3799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:34.095584 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610957c3-da18-44a6-afb9-0b3c5cdb3799 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:34.096310 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610957c3-da18-44a6-afb9-0b3c5cdb3799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:34.097900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610957c3-da18-44a6-afb9-0b3c5cdb3799 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:34.099113 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:34.110489 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1034/2080] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:22:34.113247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3814f-06e5-44ab-8237-e25ab9b5daf5 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:34.124390 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3814f-06e5-44ab-8237-e25ab9b5daf5 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:34.129060 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc887aa6-cccb-4433-a9c8-aa43d9f7e3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:34.129969 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc887aa6-cccb-4433-a9c8-aa43d9f7e3b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:34.130249 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc887aa6-cccb-4433-a9c8-aa43d9f7e3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:34.130519 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc887aa6-cccb-4433-a9c8-aa43d9f7e3b8 admin admin] PATH_INFO: `/v3/projects/7b3a6c1570e64ea786354a7f37a4acea/users/d93bfad823684f438dd7ef6078219547/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:34.132530 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc887aa6-cccb-4433-a9c8-aa43d9f7e3b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b3a6c1570e64ea786354a7f37a4acea, user_id=d93bfad823684f438dd7ef6078219547, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:34.135152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc887aa6-cccb-4433-a9c8-aa43d9f7e3b8 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:34.145588 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1047/2081] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:34 2022] PUT /identity/v3/projects/7b3a6c1570e64ea786354a7f37a4acea/users/d93bfad823684f438dd7ef6078219547/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:34.155926 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417554cf-8530-488a-87ba-f2a28c85d90a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:34.156300 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417554cf-8530-488a-87ba-f2a28c85d90a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:34.156652 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417554cf-8530-488a-87ba-f2a28c85d90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:34.184375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-417554cf-8530-488a-87ba-f2a28c85d90a None None] MFA Rules not processed for user `d93bfad823684f438dd7ef6078219547`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:34.186466 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-417554cf-8530-488a-87ba-f2a28c85d90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:34.267468 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1035/2082] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:34 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:22:34.278312 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc887aa6-cccb-4433-a9c8-aa43d9f7e3b8 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:34.280065 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-dc887aa6-cccb-4433-a9c8-aa43d9f7e3b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:34.336534 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc887aa6-cccb-4433-a9c8-aa43d9f7e3b8 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:34.340116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad301af-82ee-4777-8f35-400820929e89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:34.340749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad301af-82ee-4777-8f35-400820929e89 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:34.341051 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad301af-82ee-4777-8f35-400820929e89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:34.341323 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad301af-82ee-4777-8f35-400820929e89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:34.350519 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad301af-82ee-4777-8f35-400820929e89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:34.351954 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad301af-82ee-4777-8f35-400820929e89 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:34.361790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1048/2083] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:34 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:34.412297 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417554cf-8530-488a-87ba-f2a28c85d90a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:34.414018 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-417554cf-8530-488a-87ba-f2a28c85d90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:34.428371 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417554cf-8530-488a-87ba-f2a28c85d90a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:34.432011 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468363c0-ec48-48e6-8f00-adbe33f7d169 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:34.432662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468363c0-ec48-48e6-8f00-adbe33f7d169 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:34.432942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468363c0-ec48-48e6-8f00-adbe33f7d169 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:34.433220 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468363c0-ec48-48e6-8f00-adbe33f7d169 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:34.489439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468363c0-ec48-48e6-8f00-adbe33f7d169 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:34.490862 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468363c0-ec48-48e6-8f00-adbe33f7d169 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:34.500393 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1036/2084] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:44.357463 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=49,keystone:INSERT=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:44.495566 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=64 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:45.112542 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad301af-82ee-4777-8f35-400820929e89 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:45.114152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-bad301af-82ee-4777-8f35-400820929e89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:45.196866 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad301af-82ee-4777-8f35-400820929e89 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:45.201490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b05c9-1c5d-4848-8f18-a2db6dffcdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:45.202436 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b05c9-1c5d-4848-8f18-a2db6dffcdae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:45.203057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b05c9-1c5d-4848-8f18-a2db6dffcdae admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:45.203183 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b05c9-1c5d-4848-8f18-a2db6dffcdae admin admin] PATH_INFO: `/v3/users/fa05e385ff9a4f298dec53fe60da840d` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:45.204448 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b05c9-1c5d-4848-8f18-a2db6dffcdae admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa05e385ff9a4f298dec53fe60da840d)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:45.206653 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b05c9-1c5d-4848-8f18-a2db6dffcdae admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:45.257794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-566b05c9-1c5d-4848-8f18-a2db6dffcdae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa05e385ff9a4f298dec53fe60da840d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:45.274880 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-566b05c9-1c5d-4848-8f18-a2db6dffcdae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa05e385ff9a4f298dec53fe60da840d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:45.275764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-566b05c9-1c5d-4848-8f18-a2db6dffcdae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa05e385ff9a4f298dec53fe60da840d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:45.276688 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-566b05c9-1c5d-4848-8f18-a2db6dffcdae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa05e385ff9a4f298dec53fe60da840d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:45.297179 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-566b05c9-1c5d-4848-8f18-a2db6dffcdae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa05e385ff9a4f298dec53fe60da840d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:45.317076 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1049/2085] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:45 2022] DELETE /identity/v3/users/fa05e385ff9a4f298dec53fe60da840d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:45.322440 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa12748-9194-4d85-884d-a85f8d7f0a16 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:45.322736 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa12748-9194-4d85-884d-a85f8d7f0a16 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:45.323036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa12748-9194-4d85-884d-a85f8d7f0a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:45.352539 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6fa12748-9194-4d85-884d-a85f8d7f0a16 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:45.354149 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6fa12748-9194-4d85-884d-a85f8d7f0a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:45.457096 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1037/2086] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:22:45.476703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b05c9-1c5d-4848-8f18-a2db6dffcdae admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:45.478434 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-566b05c9-1c5d-4848-8f18-a2db6dffcdae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:45.574293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b05c9-1c5d-4848-8f18-a2db6dffcdae admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:45.582563 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628b96c-3340-43bf-93cb-33bb93201c53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:45.583255 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628b96c-3340-43bf-93cb-33bb93201c53 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:45.583544 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628b96c-3340-43bf-93cb-33bb93201c53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:45.583872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628b96c-3340-43bf-93cb-33bb93201c53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:45.598663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9628b96c-3340-43bf-93cb-33bb93201c53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:45.600461 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9628b96c-3340-43bf-93cb-33bb93201c53 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:45.612765 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1050/2087] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:45.905187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa12748-9194-4d85-884d-a85f8d7f0a16 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:45.906762 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6fa12748-9194-4d85-884d-a85f8d7f0a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:45.918601 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa12748-9194-4d85-884d-a85f8d7f0a16 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:45.923677 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6f440-8f75-4408-befc-2300c8304299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:45.924664 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6f440-8f75-4408-befc-2300c8304299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:45.925024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6f440-8f75-4408-befc-2300c8304299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:45.925303 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6f440-8f75-4408-befc-2300c8304299 admin admin] PATH_INFO: `/v3/projects/1ac42a9df13d40b98919d72a5e65ce62` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:45.927053 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc6f440-8f75-4408-befc-2300c8304299 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ac42a9df13d40b98919d72a5e65ce62)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:45.928994 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc6f440-8f75-4408-befc-2300c8304299 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:45.976731 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9dc6f440-8f75-4408-befc-2300c8304299 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ac42a9df13d40b98919d72a5e65ce62', 'request_id': 'req-9dc6f440-8f75-4408-befc-2300c8304299'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:45.991473 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9dc6f440-8f75-4408-befc-2300c8304299 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ac42a9df13d40b98919d72a5e65ce62', 'request_id': 'req-9dc6f440-8f75-4408-befc-2300c8304299'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:45.998053 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9dc6f440-8f75-4408-befc-2300c8304299 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ac42a9df13d40b98919d72a5e65ce62', 'request_id': 'req-9dc6f440-8f75-4408-befc-2300c8304299'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:46.009528 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9dc6f440-8f75-4408-befc-2300c8304299 admin admin] The token cache is being invalidate because project 1ac42a9df13d40b98919d72a5e65ce62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:46.009840 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9dc6f440-8f75-4408-befc-2300c8304299 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:46.010522 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9dc6f440-8f75-4408-befc-2300c8304299 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:46.012323 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1038/2088] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:45 2022] DELETE /identity/v3/projects/1ac42a9df13d40b98919d72a5e65ce62 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:46.016673 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628b96c-3340-43bf-93cb-33bb93201c53 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:46.026581 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628b96c-3340-43bf-93cb-33bb93201c53 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:46.049008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:46.092547 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72c5af-de8e-479c-83e0-accb0c4c533c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:46.093347 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72c5af-de8e-479c-83e0-accb0c4c533c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:46.093629 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72c5af-de8e-479c-83e0-accb0c4c533c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:46.093902 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72c5af-de8e-479c-83e0-accb0c4c533c admin admin] PATH_INFO: `/v3/users/37011663448742b091c2bc451dddafbf` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:46.095108 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72c5af-de8e-479c-83e0-accb0c4c533c admin admin] RBAC: Authorizing `identity:delete_user(user_id=37011663448742b091c2bc451dddafbf)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:46.097080 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72c5af-de8e-479c-83e0-accb0c4c533c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:46.148044 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-5e72c5af-de8e-479c-83e0-accb0c4c533c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37011663448742b091c2bc451dddafbf'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:46.169462 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-5e72c5af-de8e-479c-83e0-accb0c4c533c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37011663448742b091c2bc451dddafbf'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:46.170433 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-5e72c5af-de8e-479c-83e0-accb0c4c533c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37011663448742b091c2bc451dddafbf'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:46.171671 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-5e72c5af-de8e-479c-83e0-accb0c4c533c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37011663448742b091c2bc451dddafbf'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:46.182230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-5e72c5af-de8e-479c-83e0-accb0c4c533c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37011663448742b091c2bc451dddafbf'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:46.199397 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1051/2089] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:46 2022] DELETE /identity/v3/users/37011663448742b091c2bc451dddafbf => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:46.445831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6f440-8f75-4408-befc-2300c8304299 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:46.447381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9dc6f440-8f75-4408-befc-2300c8304299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:46.461633 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6f440-8f75-4408-befc-2300c8304299 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:46.524009 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e891220-aa7b-4ee4-87f0-a1eea15a4c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:46.524961 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e891220-aa7b-4ee4-87f0-a1eea15a4c49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:46.525242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e891220-aa7b-4ee4-87f0-a1eea15a4c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:46.525516 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e891220-aa7b-4ee4-87f0-a1eea15a4c49 admin admin] PATH_INFO: `/v3/users/88f98c5b01d548b1b62f1ff903f4b7bf` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:46.526716 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e891220-aa7b-4ee4-87f0-a1eea15a4c49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88f98c5b01d548b1b62f1ff903f4b7bf)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:46.529320 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e891220-aa7b-4ee4-87f0-a1eea15a4c49 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:46.533635 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72c5af-de8e-479c-83e0-accb0c4c533c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:46.535849 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5e72c5af-de8e-479c-83e0-accb0c4c533c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:46.551112 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72c5af-de8e-479c-83e0-accb0c4c533c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:46.557078 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67938d-6fd4-4ada-bc0a-1b5cfcb7db97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:46.558592 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67938d-6fd4-4ada-bc0a-1b5cfcb7db97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:46.559005 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67938d-6fd4-4ada-bc0a-1b5cfcb7db97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:46.560085 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67938d-6fd4-4ada-bc0a-1b5cfcb7db97 admin admin] PATH_INFO: `/v3/projects/5ba58c4ae7394baea46446abc1a2d7e9` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:46.561921 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67938d-6fd4-4ada-bc0a-1b5cfcb7db97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ba58c4ae7394baea46446abc1a2d7e9)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:46.564074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67938d-6fd4-4ada-bc0a-1b5cfcb7db97 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:46.566615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:46.588872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0e891220-aa7b-4ee4-87f0-a1eea15a4c49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88f98c5b01d548b1b62f1ff903f4b7bf'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:46.589540 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0e891220-aa7b-4ee4-87f0-a1eea15a4c49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88f98c5b01d548b1b62f1ff903f4b7bf'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:46.590150 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0e891220-aa7b-4ee4-87f0-a1eea15a4c49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88f98c5b01d548b1b62f1ff903f4b7bf'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:46.598826 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0e891220-aa7b-4ee4-87f0-a1eea15a4c49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88f98c5b01d548b1b62f1ff903f4b7bf'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:46.612098 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0e891220-aa7b-4ee4-87f0-a1eea15a4c49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88f98c5b01d548b1b62f1ff903f4b7bf'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:46.614652 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cf67938d-6fd4-4ada-bc0a-1b5cfcb7db97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ba58c4ae7394baea46446abc1a2d7e9', 'request_id': 'req-cf67938d-6fd4-4ada-bc0a-1b5cfcb7db97'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:46.624627 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cf67938d-6fd4-4ada-bc0a-1b5cfcb7db97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ba58c4ae7394baea46446abc1a2d7e9', 'request_id': 'req-cf67938d-6fd4-4ada-bc0a-1b5cfcb7db97'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:46.630720 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1039/2090] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:46 2022] DELETE /identity/v3/users/88f98c5b01d548b1b62f1ff903f4b7bf => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:46.632511 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a95dfb-2c39-4269-825e-6bd65d6835c3 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:46.632639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cf67938d-6fd4-4ada-bc0a-1b5cfcb7db97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ba58c4ae7394baea46446abc1a2d7e9', 'request_id': 'req-cf67938d-6fd4-4ada-bc0a-1b5cfcb7db97'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:46.632817 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a95dfb-2c39-4269-825e-6bd65d6835c3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:46.633132 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a95dfb-2c39-4269-825e-6bd65d6835c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:46.647379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cf67938d-6fd4-4ada-bc0a-1b5cfcb7db97 admin admin] The token cache is being invalidate because project 5ba58c4ae7394baea46446abc1a2d7e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:46.647662 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cf67938d-6fd4-4ada-bc0a-1b5cfcb7db97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:46.648434 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cf67938d-6fd4-4ada-bc0a-1b5cfcb7db97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:46.653109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1052/2091] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:46 2022] DELETE /identity/v3/projects/5ba58c4ae7394baea46446abc1a2d7e9 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:22:46.654981 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-55a95dfb-2c39-4269-825e-6bd65d6835c3 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:46.656238 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-55a95dfb-2c39-4269-825e-6bd65d6835c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:46.663093 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a00e74-e754-4216-9f70-c70e945616f8 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:46.663676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a00e74-e754-4216-9f70-c70e945616f8 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:46.663973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a00e74-e754-4216-9f70-c70e945616f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:46.693378 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-59a00e74-e754-4216-9f70-c70e945616f8 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:46.694645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-59a00e74-e754-4216-9f70-c70e945616f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:46.721094 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:46.750801 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1040/2092] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:46.789842 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1053/2093] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:46.790153 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829536f-27fd-4f80-b954-44d6d8dfaa5f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:46.790153 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829536f-27fd-4f80-b954-44d6d8dfaa5f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:46.790338 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829536f-27fd-4f80-b954-44d6d8dfaa5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:46.792339 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a00e74-e754-4216-9f70-c70e945616f8 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:46.794540 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-59a00e74-e754-4216-9f70-c70e945616f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:46.829430 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f829536f-27fd-4f80-b954-44d6d8dfaa5f None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:46.830262 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f829536f-27fd-4f80-b954-44d6d8dfaa5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:46.844295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1041/2094] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:22:46.848957 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829536f-27fd-4f80-b954-44d6d8dfaa5f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:46.860556 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829536f-27fd-4f80-b954-44d6d8dfaa5f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:46.865822 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ce090-aee8-46f8-a6c4-bcd2bc83ce4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:46.866598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ce090-aee8-46f8-a6c4-bcd2bc83ce4a admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:46.866954 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ce090-aee8-46f8-a6c4-bcd2bc83ce4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:46.867382 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ce090-aee8-46f8-a6c4-bcd2bc83ce4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:46.868250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ce090-aee8-46f8-a6c4-bcd2bc83ce4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:22:46.868724 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ce090-aee8-46f8-a6c4-bcd2bc83ce4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:46.870492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ce090-aee8-46f8-a6c4-bcd2bc83ce4a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:46.886187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1042/2095] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:22:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:22:46.888082 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a00e74-e754-4216-9f70-c70e945616f8 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:46.890997 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ce090-aee8-46f8-a6c4-bcd2bc83ce4a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:46.899802 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87830c6e-706c-408c-ab19-b6ef9d49b56d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:46.900480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87830c6e-706c-408c-ab19-b6ef9d49b56d service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:46.900819 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87830c6e-706c-408c-ab19-b6ef9d49b56d service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:46.901104 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87830c6e-706c-408c-ab19-b6ef9d49b56d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:46.924659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ce090-aee8-46f8-a6c4-bcd2bc83ce4a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:46.925086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87830c6e-706c-408c-ab19-b6ef9d49b56d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:46.925086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87830c6e-706c-408c-ab19-b6ef9d49b56d service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:46.925226 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cd0cf-460f-4f81-9f86-376c1ca890f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:46.933292 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1054/2096] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:46.933744 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cd0cf-460f-4f81-9f86-376c1ca890f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:46.934041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cd0cf-460f-4f81-9f86-376c1ca890f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:46.934503 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cd0cf-460f-4f81-9f86-376c1ca890f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:46.935368 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226cd0cf-460f-4f81-9f86-376c1ca890f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:46.937408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226cd0cf-460f-4f81-9f86-376c1ca890f4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:46.990511 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1043/2097] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:46 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:46.996207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87830c6e-706c-408c-ab19-b6ef9d49b56d service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:47.013729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87830c6e-706c-408c-ab19-b6ef9d49b56d service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:47.131153 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450026c-56bf-4189-aecb-0e6f1862591f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:47.132671 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450026c-56bf-4189-aecb-0e6f1862591f admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:47.133460 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450026c-56bf-4189-aecb-0e6f1862591f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:47.134022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450026c-56bf-4189-aecb-0e6f1862591f admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:47.134979 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450026c-56bf-4189-aecb-0e6f1862591f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:47.138364 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450026c-56bf-4189-aecb-0e6f1862591f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:47.183298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1055/2098] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:46 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:47.188027 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cd0cf-460f-4f81-9f86-376c1ca890f4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:47.203108 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cd0cf-460f-4f81-9f86-376c1ca890f4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:47.214148 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f6ca8-6d1d-4a6c-886e-27433088cebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:47.214920 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f6ca8-6d1d-4a6c-886e-27433088cebc admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:47.215137 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f6ca8-6d1d-4a6c-886e-27433088cebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:47.215427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f6ca8-6d1d-4a6c-886e-27433088cebc admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:47.216613 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f6ca8-6d1d-4a6c-886e-27433088cebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:47.218416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f6ca8-6d1d-4a6c-886e-27433088cebc admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:47.234039 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1044/2099] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:47.235085 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450026c-56bf-4189-aecb-0e6f1862591f admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:47.245879 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450026c-56bf-4189-aecb-0e6f1862591f admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:47.252922 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c6d21-ebb0-413a-bc5a-9f8b66770c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:47.253584 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c6d21-ebb0-413a-bc5a-9f8b66770c39 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:47.254912 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c6d21-ebb0-413a-bc5a-9f8b66770c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:47.255131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c6d21-ebb0-413a-bc5a-9f8b66770c39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:47.256106 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0c6d21-ebb0-413a-bc5a-9f8b66770c39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:47.258008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0c6d21-ebb0-413a-bc5a-9f8b66770c39 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:47.275661 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f6ca8-6d1d-4a6c-886e-27433088cebc admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:47.282819 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1056/2100] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:47 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:22:47.287581 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f6ca8-6d1d-4a6c-886e-27433088cebc admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:47.293518 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf9ba9-59e1-42c6-9586-13b9e1300d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:47.294524 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf9ba9-59e1-42c6-9586-13b9e1300d08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:47.294737 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf9ba9-59e1-42c6-9586-13b9e1300d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:47.295183 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf9ba9-59e1-42c6-9586-13b9e1300d08 admin admin] PATH_INFO: `/v3/projects/56f7125d3d184037b5332527b70a9a71/users/b2841cacb67646d5a4954d2ac95953a7/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:47.315097 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf9ba9-59e1-42c6-9586-13b9e1300d08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56f7125d3d184037b5332527b70a9a71, user_id=b2841cacb67646d5a4954d2ac95953a7, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:47.321513 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf9ba9-59e1-42c6-9586-13b9e1300d08 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:47.322619 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c6d21-ebb0-413a-bc5a-9f8b66770c39 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:47.324656 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9b0c6d21-ebb0-413a-bc5a-9f8b66770c39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:47.340974 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c6d21-ebb0-413a-bc5a-9f8b66770c39 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:47.345722 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1045/2101] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:47 2022] PUT /identity/v3/projects/56f7125d3d184037b5332527b70a9a71/users/b2841cacb67646d5a4954d2ac95953a7/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:22:47.357345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fd2c0-31f1-48f3-b7b6-46ef620c3d75 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:47.357531 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fd2c0-31f1-48f3-b7b6-46ef620c3d75 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:47.357850 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fd2c0-31f1-48f3-b7b6-46ef620c3d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:47.393033 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ce4fd2c0-31f1-48f3-b7b6-46ef620c3d75 None None] MFA Rules not processed for user `b2841cacb67646d5a4954d2ac95953a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:47.394441 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ce4fd2c0-31f1-48f3-b7b6-46ef620c3d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:47.397944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:47.413456 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:47.415818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ef081-642d-403d-939a-84b91ce3fce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:47.416911 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ef081-642d-403d-939a-84b91ce3fce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:47.417384 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ef081-642d-403d-939a-84b91ce3fce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:47.417816 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ef081-642d-403d-939a-84b91ce3fce0 admin admin] PATH_INFO: `/v3/projects/1f61c91a015645b0a4722285025a67d7` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:47.419353 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ef081-642d-403d-939a-84b91ce3fce0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f61c91a015645b0a4722285025a67d7)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:47.421438 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ef081-642d-403d-939a-84b91ce3fce0 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:47.490887 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1046/2102] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:22:47.501431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fd2c0-31f1-48f3-b7b6-46ef620c3d75 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:47.502870 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ce4fd2c0-31f1-48f3-b7b6-46ef620c3d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:47.503417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-157ef081-642d-403d-939a-84b91ce3fce0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f61c91a015645b0a4722285025a67d7', 'request_id': 'req-157ef081-642d-403d-939a-84b91ce3fce0'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:47.513566 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-157ef081-642d-403d-939a-84b91ce3fce0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f61c91a015645b0a4722285025a67d7', 'request_id': 'req-157ef081-642d-403d-939a-84b91ce3fce0'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:47.521786 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-157ef081-642d-403d-939a-84b91ce3fce0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f61c91a015645b0a4722285025a67d7', 'request_id': 'req-157ef081-642d-403d-939a-84b91ce3fce0'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:47.536901 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-157ef081-642d-403d-939a-84b91ce3fce0 admin admin] The token cache is being invalidate because project 1f61c91a015645b0a4722285025a67d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:47.537594 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-157ef081-642d-403d-939a-84b91ce3fce0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:47.538321 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-157ef081-642d-403d-939a-84b91ce3fce0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:47.540135 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1057/2103] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:47 2022] DELETE /identity/v3/projects/1f61c91a015645b0a4722285025a67d7 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:22:47.545083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ef081-642d-403d-939a-84b91ce3fce0 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:47.546757 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-157ef081-642d-403d-939a-84b91ce3fce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:47.570836 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fd2c0-31f1-48f3-b7b6-46ef620c3d75 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:47.618143 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ef081-642d-403d-939a-84b91ce3fce0 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:47.623854 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f132a-c879-4e02-98d9-effe90c6ea9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:47.624731 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f132a-c879-4e02-98d9-effe90c6ea9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:47.625011 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f132a-c879-4e02-98d9-effe90c6ea9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:47.625114 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803bef5-ac5a-4e4a-ba50-2fb9693915ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:47.625337 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f132a-c879-4e02-98d9-effe90c6ea9b admin admin] PATH_INFO: `/v3/users/d93bfad823684f438dd7ef6078219547` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:47.625751 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803bef5-ac5a-4e4a-ba50-2fb9693915ef service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:47.626037 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803bef5-ac5a-4e4a-ba50-2fb9693915ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:47.626314 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803bef5-ac5a-4e4a-ba50-2fb9693915ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:47.626638 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854f132a-c879-4e02-98d9-effe90c6ea9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d93bfad823684f438dd7ef6078219547)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:47.628146 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b803bef5-ac5a-4e4a-ba50-2fb9693915ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:47.628751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854f132a-c879-4e02-98d9-effe90c6ea9b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:47.678053 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-854f132a-c879-4e02-98d9-effe90c6ea9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd93bfad823684f438dd7ef6078219547'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:47.692548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b803bef5-ac5a-4e4a-ba50-2fb9693915ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:47.693959 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b803bef5-ac5a-4e4a-ba50-2fb9693915ef service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:47.696750 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-854f132a-c879-4e02-98d9-effe90c6ea9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd93bfad823684f438dd7ef6078219547'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:47.697485 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-854f132a-c879-4e02-98d9-effe90c6ea9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd93bfad823684f438dd7ef6078219547'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:47.698222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-854f132a-c879-4e02-98d9-effe90c6ea9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd93bfad823684f438dd7ef6078219547'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:47.704383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1047/2104] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:47.707689 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-854f132a-c879-4e02-98d9-effe90c6ea9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd93bfad823684f438dd7ef6078219547'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:47.711566 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:47.726917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803bef5-ac5a-4e4a-ba50-2fb9693915ef service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:47.728703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b803bef5-ac5a-4e4a-ba50-2fb9693915ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:47.733210 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1058/2105] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:47 2022] DELETE /identity/v3/users/d93bfad823684f438dd7ef6078219547 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:22:47.744904 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803bef5-ac5a-4e4a-ba50-2fb9693915ef service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:47.771342 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:47.808414 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadfc84d-cf09-469d-b599-1d61ac3e175b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:47.809458 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadfc84d-cf09-469d-b599-1d61ac3e175b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:47.809881 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadfc84d-cf09-469d-b599-1d61ac3e175b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:47.810163 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadfc84d-cf09-469d-b599-1d61ac3e175b admin admin] PATH_INFO: `/v3/users/b2841cacb67646d5a4954d2ac95953a7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:47.811890 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadfc84d-cf09-469d-b599-1d61ac3e175b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2841cacb67646d5a4954d2ac95953a7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:47.813853 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadfc84d-cf09-469d-b599-1d61ac3e175b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:47.866418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dadfc84d-cf09-469d-b599-1d61ac3e175b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2841cacb67646d5a4954d2ac95953a7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:47.867149 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dadfc84d-cf09-469d-b599-1d61ac3e175b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2841cacb67646d5a4954d2ac95953a7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:47.867867 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dadfc84d-cf09-469d-b599-1d61ac3e175b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2841cacb67646d5a4954d2ac95953a7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:47.876505 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dadfc84d-cf09-469d-b599-1d61ac3e175b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2841cacb67646d5a4954d2ac95953a7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:47.901596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dadfc84d-cf09-469d-b599-1d61ac3e175b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2841cacb67646d5a4954d2ac95953a7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:47.916735 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1048/2106] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:47 2022] DELETE /identity/v3/users/b2841cacb67646d5a4954d2ac95953a7 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:22:47.921231 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173ba25-8345-4cee-a619-9dd20da7180f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:47.921518 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173ba25-8345-4cee-a619-9dd20da7180f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:47.921963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173ba25-8345-4cee-a619-9dd20da7180f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:47.943723 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-c173ba25-8345-4cee-a619-9dd20da7180f None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:47.945042 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c173ba25-8345-4cee-a619-9dd20da7180f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:48.019010 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1059/2107] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:48.028965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadfc84d-cf09-469d-b599-1d61ac3e175b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:48.031004 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dadfc84d-cf09-469d-b599-1d61ac3e175b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:48.094884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadfc84d-cf09-469d-b599-1d61ac3e175b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:48.098941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0714e-4e10-4519-a19f-111b9b1c0dfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:48.099882 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0714e-4e10-4519-a19f-111b9b1c0dfe service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:48.100166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0714e-4e10-4519-a19f-111b9b1c0dfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:48.100440 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0714e-4e10-4519-a19f-111b9b1c0dfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:48.110170 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c173ba25-8345-4cee-a619-9dd20da7180f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:48.110580 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0714e-4e10-4519-a19f-111b9b1c0dfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:48.112055 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c173ba25-8345-4cee-a619-9dd20da7180f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:48.112186 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0714e-4e10-4519-a19f-111b9b1c0dfe service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:48.122560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1049/2108] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:48.125641 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c173ba25-8345-4cee-a619-9dd20da7180f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:48.130783 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc588ca-1925-4bdc-99d6-65b25355e8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:48.131874 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc588ca-1925-4bdc-99d6-65b25355e8ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:48.132783 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc588ca-1925-4bdc-99d6-65b25355e8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:48.133124 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc588ca-1925-4bdc-99d6-65b25355e8ee admin admin] PATH_INFO: `/v3/projects/7b3a6c1570e64ea786354a7f37a4acea` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:48.134734 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc588ca-1925-4bdc-99d6-65b25355e8ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b3a6c1570e64ea786354a7f37a4acea)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:48.136896 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc588ca-1925-4bdc-99d6-65b25355e8ee admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:48.222997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9dc588ca-1925-4bdc-99d6-65b25355e8ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b3a6c1570e64ea786354a7f37a4acea', 'request_id': 'req-9dc588ca-1925-4bdc-99d6-65b25355e8ee'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:48.232872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9dc588ca-1925-4bdc-99d6-65b25355e8ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b3a6c1570e64ea786354a7f37a4acea', 'request_id': 'req-9dc588ca-1925-4bdc-99d6-65b25355e8ee'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:48.244103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9dc588ca-1925-4bdc-99d6-65b25355e8ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b3a6c1570e64ea786354a7f37a4acea', 'request_id': 'req-9dc588ca-1925-4bdc-99d6-65b25355e8ee'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:48.261460 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9dc588ca-1925-4bdc-99d6-65b25355e8ee admin admin] The token cache is being invalidate because project 7b3a6c1570e64ea786354a7f37a4acea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:48.261818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9dc588ca-1925-4bdc-99d6-65b25355e8ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:48.263522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9dc588ca-1925-4bdc-99d6-65b25355e8ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:48.270639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1060/2109] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:48 2022] DELETE /identity/v3/projects/7b3a6c1570e64ea786354a7f37a4acea => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:22:48.273040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9178a10-9a0f-430c-a7ff-482c54fd2102 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:48.273327 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9178a10-9a0f-430c-a7ff-482c54fd2102 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:48.273628 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9178a10-9a0f-430c-a7ff-482c54fd2102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:48.313780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b9178a10-9a0f-430c-a7ff-482c54fd2102 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:48.323987 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b9178a10-9a0f-430c-a7ff-482c54fd2102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:48.358291 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc588ca-1925-4bdc-99d6-65b25355e8ee admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:48.359974 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9dc588ca-1925-4bdc-99d6-65b25355e8ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:48.392264 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:48.431423 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1050/2110] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:48.434018 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d038e6-4032-4c19-9ee1-ec3e35ebb508 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:48.434018 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d038e6-4032-4c19-9ee1-ec3e35ebb508 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:48.434018 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d038e6-4032-4c19-9ee1-ec3e35ebb508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:48.464336 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-70d038e6-4032-4c19-9ee1-ec3e35ebb508 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:48.466535 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-70d038e6-4032-4c19-9ee1-ec3e35ebb508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:48.477098 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1051/2111] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:48.480535 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc588ca-1925-4bdc-99d6-65b25355e8ee admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:48.481326 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d038e6-4032-4c19-9ee1-ec3e35ebb508 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:48.487572 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ec6c9-93a1-4374-ad71-b06984f9eb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:48.488831 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ec6c9-93a1-4374-ad71-b06984f9eb24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:48.489333 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ec6c9-93a1-4374-ad71-b06984f9eb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:48.489811 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ec6c9-93a1-4374-ad71-b06984f9eb24 admin admin] PATH_INFO: `/v3/users/2385dea59da944b693007bee2a880433` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:48.491290 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ec6c9-93a1-4374-ad71-b06984f9eb24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2385dea59da944b693007bee2a880433)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:48.491568 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d038e6-4032-4c19-9ee1-ec3e35ebb508 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:48.494031 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ec6c9-93a1-4374-ad71-b06984f9eb24 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:48.504030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5423c69-e53a-4d97-99bc-58bf17ad2238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:48.505048 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5423c69-e53a-4d97-99bc-58bf17ad2238 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:48.505333 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5423c69-e53a-4d97-99bc-58bf17ad2238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:48.505605 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5423c69-e53a-4d97-99bc-58bf17ad2238 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:48.507068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5423c69-e53a-4d97-99bc-58bf17ad2238 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:22:48.507409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5423c69-e53a-4d97-99bc-58bf17ad2238 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:48.509020 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5423c69-e53a-4d97-99bc-58bf17ad2238 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:48.528574 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1052/2112] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:22:48.529671 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5423c69-e53a-4d97-99bc-58bf17ad2238 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:48.532008 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e5423c69-e53a-4d97-99bc-58bf17ad2238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:48.536655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:48.564842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5423c69-e53a-4d97-99bc-58bf17ad2238 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:48.583774 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e2b5a8-0d2c-41ba-a1cb-31614ab8bbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:48.585378 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2b5a8-0d2c-41ba-a1cb-31614ab8bbd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:48.585696 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2b5a8-0d2c-41ba-a1cb-31614ab8bbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:48.586141 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2b5a8-0d2c-41ba-a1cb-31614ab8bbd6 admin admin] PATH_INFO: `/v3/projects/56f7125d3d184037b5332527b70a9a71` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:48.587595 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e2b5a8-0d2c-41ba-a1cb-31614ab8bbd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56f7125d3d184037b5332527b70a9a71)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:48.592270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e2b5a8-0d2c-41ba-a1cb-31614ab8bbd6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:48.595944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ed9ec6c9-93a1-4374-ad71-b06984f9eb24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2385dea59da944b693007bee2a880433'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:48.624220 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ed9ec6c9-93a1-4374-ad71-b06984f9eb24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2385dea59da944b693007bee2a880433'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:48.624975 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ed9ec6c9-93a1-4374-ad71-b06984f9eb24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2385dea59da944b693007bee2a880433'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:48.626420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ed9ec6c9-93a1-4374-ad71-b06984f9eb24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2385dea59da944b693007bee2a880433'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:48.639492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ed9ec6c9-93a1-4374-ad71-b06984f9eb24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2385dea59da944b693007bee2a880433'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:48.662228 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1061/2113] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:48 2022] DELETE /identity/v3/users/2385dea59da944b693007bee2a880433 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:48.663329 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ec6c9-93a1-4374-ad71-b06984f9eb24 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:48.665379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ed9ec6c9-93a1-4374-ad71-b06984f9eb24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:48.665584 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e2e2b5a8-0d2c-41ba-a1cb-31614ab8bbd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56f7125d3d184037b5332527b70a9a71', 'request_id': 'req-e2e2b5a8-0d2c-41ba-a1cb-31614ab8bbd6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:48.676835 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e2e2b5a8-0d2c-41ba-a1cb-31614ab8bbd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56f7125d3d184037b5332527b70a9a71', 'request_id': 'req-e2e2b5a8-0d2c-41ba-a1cb-31614ab8bbd6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:48.684114 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e2e2b5a8-0d2c-41ba-a1cb-31614ab8bbd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56f7125d3d184037b5332527b70a9a71', 'request_id': 'req-e2e2b5a8-0d2c-41ba-a1cb-31614ab8bbd6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:48.704042 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e2e2b5a8-0d2c-41ba-a1cb-31614ab8bbd6 admin admin] The token cache is being invalidate because project 56f7125d3d184037b5332527b70a9a71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:48.704588 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e2e2b5a8-0d2c-41ba-a1cb-31614ab8bbd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:48.705468 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e2e2b5a8-0d2c-41ba-a1cb-31614ab8bbd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:48.707394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1053/2114] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:48 2022] DELETE /identity/v3/projects/56f7125d3d184037b5332527b70a9a71 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:48.709078 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa40876-7174-4e68-b46d-88ffa7724262 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:48.709517 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa40876-7174-4e68-b46d-88ffa7724262 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:48.709949 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa40876-7174-4e68-b46d-88ffa7724262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:48.730761 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5aa40876-7174-4e68-b46d-88ffa7724262 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:48.733281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5aa40876-7174-4e68-b46d-88ffa7724262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:48.783974 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ec6c9-93a1-4374-ad71-b06984f9eb24 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:48.795674 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35fb46-41f9-4743-8202-07897f0edd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:48.796135 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35fb46-41f9-4743-8202-07897f0edd9c admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:48.796426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35fb46-41f9-4743-8202-07897f0edd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:48.798238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35fb46-41f9-4743-8202-07897f0edd9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:48.800704 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35fb46-41f9-4743-8202-07897f0edd9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:48.804976 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35fb46-41f9-4743-8202-07897f0edd9c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:48.839054 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1054/2115] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:48.841246 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e546ef-355c-41fa-8f47-1c96a3a782cb None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:48.841767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e546ef-355c-41fa-8f47-1c96a3a782cb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:48.842072 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e546ef-355c-41fa-8f47-1c96a3a782cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:48.853550 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1062/2116] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:48 2022] POST /identity/v3/projects => generated 399 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:48.858294 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35fb46-41f9-4743-8202-07897f0edd9c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:48.860221 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-cd35fb46-41f9-4743-8202-07897f0edd9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:48.863913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-81e546ef-355c-41fa-8f47-1c96a3a782cb None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:48.865286 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-81e546ef-355c-41fa-8f47-1c96a3a782cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:48.922602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:48.938925 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1055/2117] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:48.939197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35fb46-41f9-4743-8202-07897f0edd9c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:48.944237 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d3be6-332d-4555-b50b-e08803a82666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:48.945171 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d3be6-332d-4555-b50b-e08803a82666 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:48.945455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d3be6-332d-4555-b50b-e08803a82666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:48.945733 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d3be6-332d-4555-b50b-e08803a82666 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:48.945826 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e546ef-355c-41fa-8f47-1c96a3a782cb None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:48.946481 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d3be6-332d-4555-b50b-e08803a82666 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:48.947181 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-81e546ef-355c-41fa-8f47-1c96a3a782cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:48.948439 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d3be6-332d-4555-b50b-e08803a82666 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:48.977395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1063/2118] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:48 2022] POST /identity/v3/users => generated 300 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:22:48.979108 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b6a76-1b03-4d32-870a-87a5f8dfeac7 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:48.979412 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b6a76-1b03-4d32-870a-87a5f8dfeac7 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:48.979675 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b6a76-1b03-4d32-870a-87a5f8dfeac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:49.004049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-7e8b6a76-1b03-4d32-870a-87a5f8dfeac7 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:49.005257 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7e8b6a76-1b03-4d32-870a-87a5f8dfeac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:49.014393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1064/2119] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:49.018392 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b6a76-1b03-4d32-870a-87a5f8dfeac7 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:49.022843 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e546ef-355c-41fa-8f47-1c96a3a782cb None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:49.027875 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f723283-5a8f-40a2-9f87-18d50e1a0fa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:49.028648 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b6a76-1b03-4d32-870a-87a5f8dfeac7 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:49.028979 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f723283-5a8f-40a2-9f87-18d50e1a0fa4 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:49.029473 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f723283-5a8f-40a2-9f87-18d50e1a0fa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:49.030023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f723283-5a8f-40a2-9f87-18d50e1a0fa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:49.034810 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd103b1-50c4-41e4-ad63-29bbfa1c5d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:49.035486 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd103b1-50c4-41e4-ad63-29bbfa1c5d77 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:49.036322 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd103b1-50c4-41e4-ad63-29bbfa1c5d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:49.036608 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd103b1-50c4-41e4-ad63-29bbfa1c5d77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:49.037346 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd103b1-50c4-41e4-ad63-29bbfa1c5d77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:49.039176 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd103b1-50c4-41e4-ad63-29bbfa1c5d77 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:49.041749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:49.042970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f723283-5a8f-40a2-9f87-18d50e1a0fa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:49.045274 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f723283-5a8f-40a2-9f87-18d50e1a0fa4 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:49.062589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1056/2120] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:49.063134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f723283-5a8f-40a2-9f87-18d50e1a0fa4 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:49.067609 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1065/2121] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:49.068990 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd103b1-50c4-41e4-ad63-29bbfa1c5d77 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:49.073366 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f723283-5a8f-40a2-9f87-18d50e1a0fa4 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:49.079860 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e17c29-7087-44cc-b8d8-d8b5f0163108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:49.080567 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e17c29-7087-44cc-b8d8-d8b5f0163108 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:49.080964 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e17c29-7087-44cc-b8d8-d8b5f0163108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:49.081274 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e17c29-7087-44cc-b8d8-d8b5f0163108 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:49.082277 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e17c29-7087-44cc-b8d8-d8b5f0163108 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:22:49.082597 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e17c29-7087-44cc-b8d8-d8b5f0163108 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:49.084331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd103b1-50c4-41e4-ad63-29bbfa1c5d77 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:49.084478 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e17c29-7087-44cc-b8d8-d8b5f0163108 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:49.089565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e4d8c-5ecd-4bd4-a313-018592a45e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:49.090233 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e4d8c-5ecd-4bd4-a313-018592a45e0d admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:49.090554 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e4d8c-5ecd-4bd4-a313-018592a45e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:49.090830 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e4d8c-5ecd-4bd4-a313-018592a45e0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:49.091642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e4d8c-5ecd-4bd4-a313-018592a45e0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:49.093240 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e4d8c-5ecd-4bd4-a313-018592a45e0d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:49.100470 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1057/2122] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:22:49.103256 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e17c29-7087-44cc-b8d8-d8b5f0163108 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:49.103256 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-88e17c29-7087-44cc-b8d8-d8b5f0163108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:49.108917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1066/2123] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:49.108917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e4d8c-5ecd-4bd4-a313-018592a45e0d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:49.119301 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e4d8c-5ecd-4bd4-a313-018592a45e0d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:49.120708 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e17c29-7087-44cc-b8d8-d8b5f0163108 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:49.125287 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189a571-42f4-4def-8e15-675299ca19b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:49.125960 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189a571-42f4-4def-8e15-675299ca19b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:49.126247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189a571-42f4-4def-8e15-675299ca19b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:49.126523 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189a571-42f4-4def-8e15-675299ca19b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:49.127243 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189a571-42f4-4def-8e15-675299ca19b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:49.128062 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd2215-671f-44ec-a19c-3b9bb94291a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:49.129037 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd2215-671f-44ec-a19c-3b9bb94291a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:49.129353 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189a571-42f4-4def-8e15-675299ca19b9 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:49.129478 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd2215-671f-44ec-a19c-3b9bb94291a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:49.129671 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd2215-671f-44ec-a19c-3b9bb94291a6 admin admin] PATH_INFO: `/v3/users/936db0436820496ebf8cc741852dda10` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:49.131046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd2215-671f-44ec-a19c-3b9bb94291a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=936db0436820496ebf8cc741852dda10)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:49.133219 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd2215-671f-44ec-a19c-3b9bb94291a6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:49.158031 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1067/2124] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:49 2022] POST /identity/v3/projects => generated 361 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:49.158628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189a571-42f4-4def-8e15-675299ca19b9 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:49.214589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-67bd2215-671f-44ec-a19c-3b9bb94291a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '936db0436820496ebf8cc741852dda10'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:49.216161 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-67bd2215-671f-44ec-a19c-3b9bb94291a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '936db0436820496ebf8cc741852dda10'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:49.217350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-67bd2215-671f-44ec-a19c-3b9bb94291a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '936db0436820496ebf8cc741852dda10'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:49.239444 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-67bd2215-671f-44ec-a19c-3b9bb94291a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '936db0436820496ebf8cc741852dda10'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:49.252113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189a571-42f4-4def-8e15-675299ca19b9 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:49.256652 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-67bd2215-671f-44ec-a19c-3b9bb94291a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '936db0436820496ebf8cc741852dda10'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:49.257135 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f135ce7-6f68-41bd-bfec-d0b6df338f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:49.258040 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f135ce7-6f68-41bd-bfec-d0b6df338f23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:49.258368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f135ce7-6f68-41bd-bfec-d0b6df338f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:49.259147 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f135ce7-6f68-41bd-bfec-d0b6df338f23 admin admin] PATH_INFO: `/v3/projects/f9ed7398854449739ad5be06f7a35441/users/fe9f703e7c704f2884f010be93db638c/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:49.278122 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f135ce7-6f68-41bd-bfec-d0b6df338f23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9ed7398854449739ad5be06f7a35441, user_id=fe9f703e7c704f2884f010be93db638c, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:49.281593 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f135ce7-6f68-41bd-bfec-d0b6df338f23 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:49.292768 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1058/2125] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:49 2022] DELETE /identity/v3/users/936db0436820496ebf8cc741852dda10 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:49.295762 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd2215-671f-44ec-a19c-3b9bb94291a6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:49.297317 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-67bd2215-671f-44ec-a19c-3b9bb94291a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:49.303774 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1068/2126] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:49 2022] PUT /identity/v3/projects/f9ed7398854449739ad5be06f7a35441/users/fe9f703e7c704f2884f010be93db638c/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:49.316413 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf25f2-bcca-4bfd-9c88-e3a078d81790 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:49.316701 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf25f2-bcca-4bfd-9c88-e3a078d81790 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:49.316979 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf25f2-bcca-4bfd-9c88-e3a078d81790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:49.349877 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-18bf25f2-bcca-4bfd-9c88-e3a078d81790 None None] MFA Rules not processed for user `fe9f703e7c704f2884f010be93db638c`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:49.351875 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-18bf25f2-bcca-4bfd-9c88-e3a078d81790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:49.372791 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:49.381784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd2215-671f-44ec-a19c-3b9bb94291a6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:49.459503 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1069/2127] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:22:49.460392 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf25f2-bcca-4bfd-9c88-e3a078d81790 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:49.463721 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-18bf25f2-bcca-4bfd-9c88-e3a078d81790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:49.472544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151c264-dffc-4276-b5c4-375ae669f8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:49.473239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151c264-dffc-4276-b5c4-375ae669f8db admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:49.473526 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151c264-dffc-4276-b5c4-375ae669f8db admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:49.473799 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151c264-dffc-4276-b5c4-375ae669f8db admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:49.474770 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2151c264-dffc-4276-b5c4-375ae669f8db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:49.477444 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2151c264-dffc-4276-b5c4-375ae669f8db admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:49.504552 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1059/2128] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:49 2022] POST /identity/v3/users => generated 281 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:49.504552 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151c264-dffc-4276-b5c4-375ae669f8db admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:49.506337 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2151c264-dffc-4276-b5c4-375ae669f8db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:49.514793 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:49.519950 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151c264-dffc-4276-b5c4-375ae669f8db admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:49.525362 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cacc11-7aab-4226-bc0f-6bcafc8f6dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:49.526290 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cacc11-7aab-4226-bc0f-6bcafc8f6dea admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:49.526643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cacc11-7aab-4226-bc0f-6bcafc8f6dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:49.526990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cacc11-7aab-4226-bc0f-6bcafc8f6dea admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:49.527796 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cacc11-7aab-4226-bc0f-6bcafc8f6dea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:49.530577 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cacc11-7aab-4226-bc0f-6bcafc8f6dea admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:49.550099 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf25f2-bcca-4bfd-9c88-e3a078d81790 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:49.558543 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6066a1-bc11-4739-bcd2-95c14e60f876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:49.560696 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6066a1-bc11-4739-bcd2-95c14e60f876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:49.561125 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6066a1-bc11-4739-bcd2-95c14e60f876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:49.561869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6066a1-bc11-4739-bcd2-95c14e60f876 admin admin] PATH_INFO: `/v3/projects/4b6b0590e8ce4760a06e634b58beb312` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:49.566287 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6066a1-bc11-4739-bcd2-95c14e60f876 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b6b0590e8ce4760a06e634b58beb312)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:49.566680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1060/2129] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:49 2022] POST /identity/v3/projects => generated 399 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:49.567698 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cacc11-7aab-4226-bc0f-6bcafc8f6dea admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:49.573079 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6066a1-bc11-4739-bcd2-95c14e60f876 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:49.634996 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-dc6066a1-bc11-4739-bcd2-95c14e60f876 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b6b0590e8ce4760a06e634b58beb312', 'request_id': 'req-dc6066a1-bc11-4739-bcd2-95c14e60f876'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:49.644734 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-dc6066a1-bc11-4739-bcd2-95c14e60f876 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b6b0590e8ce4760a06e634b58beb312', 'request_id': 'req-dc6066a1-bc11-4739-bcd2-95c14e60f876'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:49.647786 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cacc11-7aab-4226-bc0f-6bcafc8f6dea admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:49.652242 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-dc6066a1-bc11-4739-bcd2-95c14e60f876 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b6b0590e8ce4760a06e634b58beb312', 'request_id': 'req-dc6066a1-bc11-4739-bcd2-95c14e60f876'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:49.663518 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-dc6066a1-bc11-4739-bcd2-95c14e60f876 admin admin] The token cache is being invalidate because project 4b6b0590e8ce4760a06e634b58beb312 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:49.663832 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-dc6066a1-bc11-4739-bcd2-95c14e60f876 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:49.664494 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-dc6066a1-bc11-4739-bcd2-95c14e60f876 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:49.666152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1070/2130] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:49 2022] DELETE /identity/v3/projects/4b6b0590e8ce4760a06e634b58beb312 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:49.666836 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6066a1-bc11-4739-bcd2-95c14e60f876 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:49.668127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-dc6066a1-bc11-4739-bcd2-95c14e60f876 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:49.708087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4afc31-4a82-4fe3-9692-5de60a5e7b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:49.709201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4afc31-4a82-4fe3-9692-5de60a5e7b66 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:49.709486 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4afc31-4a82-4fe3-9692-5de60a5e7b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:49.709764 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4afc31-4a82-4fe3-9692-5de60a5e7b66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:49.710859 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4afc31-4a82-4fe3-9692-5de60a5e7b66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:49.712490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4afc31-4a82-4fe3-9692-5de60a5e7b66 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:49.726197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1061/2131] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:49.726197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4afc31-4a82-4fe3-9692-5de60a5e7b66 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:49.726197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ea4afc31-4a82-4fe3-9692-5de60a5e7b66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:49.734362 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6066a1-bc11-4739-bcd2-95c14e60f876 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:49.738812 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4afc31-4a82-4fe3-9692-5de60a5e7b66 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:49.739009 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55fb08-d256-4cd6-9940-9a4769289f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:49.739806 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55fb08-d256-4cd6-9940-9a4769289f93 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:49.740087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55fb08-d256-4cd6-9940-9a4769289f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:49.740377 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55fb08-d256-4cd6-9940-9a4769289f93 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:49.741054 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a55fb08-d256-4cd6-9940-9a4769289f93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:49.742950 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a55fb08-d256-4cd6-9940-9a4769289f93 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:49.743620 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d6177-bf81-4fd3-8e4e-dd2bfe8c8bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:49.744454 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d6177-bf81-4fd3-8e4e-dd2bfe8c8bb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:49.744738 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d6177-bf81-4fd3-8e4e-dd2bfe8c8bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:49.745015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d6177-bf81-4fd3-8e4e-dd2bfe8c8bb8 admin admin] PATH_INFO: `/v3/projects/5da54606acd24962be630e28ff636d72` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:49.746189 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d6177-bf81-4fd3-8e4e-dd2bfe8c8bb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5da54606acd24962be630e28ff636d72)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:49.748235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d6177-bf81-4fd3-8e4e-dd2bfe8c8bb8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:49.756689 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:49.768863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1071/2132] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:49 2022] POST /identity/v3/users => generated 306 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:49.769974 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55fb08-d256-4cd6-9940-9a4769289f93 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:49.771813 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3a55fb08-d256-4cd6-9940-9a4769289f93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:49.784473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55fb08-d256-4cd6-9940-9a4769289f93 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:49.797718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8d5d6177-bf81-4fd3-8e4e-dd2bfe8c8bb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5da54606acd24962be630e28ff636d72', 'request_id': 'req-8d5d6177-bf81-4fd3-8e4e-dd2bfe8c8bb8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:49.804648 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8d5d6177-bf81-4fd3-8e4e-dd2bfe8c8bb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5da54606acd24962be630e28ff636d72', 'request_id': 'req-8d5d6177-bf81-4fd3-8e4e-dd2bfe8c8bb8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:49.810834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8d5d6177-bf81-4fd3-8e4e-dd2bfe8c8bb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5da54606acd24962be630e28ff636d72', 'request_id': 'req-8d5d6177-bf81-4fd3-8e4e-dd2bfe8c8bb8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:49.826489 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8d5d6177-bf81-4fd3-8e4e-dd2bfe8c8bb8 admin admin] The token cache is being invalidate because project 5da54606acd24962be630e28ff636d72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:49.826783 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8d5d6177-bf81-4fd3-8e4e-dd2bfe8c8bb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:49.827512 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8d5d6177-bf81-4fd3-8e4e-dd2bfe8c8bb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:49.829378 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1062/2133] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:49 2022] DELETE /identity/v3/projects/5da54606acd24962be630e28ff636d72 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:49.830203 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d6177-bf81-4fd3-8e4e-dd2bfe8c8bb8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:49.835622 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8d5d6177-bf81-4fd3-8e4e-dd2bfe8c8bb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:49.842905 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:49.851255 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786b20e-6fe9-4e09-87fc-8ab955f4ec9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:49.852281 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786b20e-6fe9-4e09-87fc-8ab955f4ec9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:49.852809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786b20e-6fe9-4e09-87fc-8ab955f4ec9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:49.853173 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786b20e-6fe9-4e09-87fc-8ab955f4ec9e admin admin] PATH_INFO: `/v3/users/6c01b5f3060341329afd16bb093dc8de` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:49.854637 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786b20e-6fe9-4e09-87fc-8ab955f4ec9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c01b5f3060341329afd16bb093dc8de)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:49.857529 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786b20e-6fe9-4e09-87fc-8ab955f4ec9e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:49.908107 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-8786b20e-6fe9-4e09-87fc-8ab955f4ec9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c01b5f3060341329afd16bb093dc8de'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:49.920646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-8786b20e-6fe9-4e09-87fc-8ab955f4ec9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c01b5f3060341329afd16bb093dc8de'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:49.921273 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-8786b20e-6fe9-4e09-87fc-8ab955f4ec9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c01b5f3060341329afd16bb093dc8de'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:49.921891 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-8786b20e-6fe9-4e09-87fc-8ab955f4ec9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c01b5f3060341329afd16bb093dc8de'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:49.921994 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d6177-bf81-4fd3-8e4e-dd2bfe8c8bb8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:49.926874 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b0be5-57ac-49e4-9306-954313da4344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:49.927650 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b0be5-57ac-49e4-9306-954313da4344 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:49.927933 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b0be5-57ac-49e4-9306-954313da4344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:49.928227 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b0be5-57ac-49e4-9306-954313da4344 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:49.929023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b0be5-57ac-49e4-9306-954313da4344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:49.929132 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-8786b20e-6fe9-4e09-87fc-8ab955f4ec9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c01b5f3060341329afd16bb093dc8de'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:49.930780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b0be5-57ac-49e4-9306-954313da4344 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:49.941790 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1063/2134] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:49.942601 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b0be5-57ac-49e4-9306-954313da4344 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:49.944720 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1072/2135] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:49 2022] DELETE /identity/v3/users/6c01b5f3060341329afd16bb093dc8de => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:49.946878 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6f8d0-3e5b-4dcc-a95e-58483dec7976 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:49.947317 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6f8d0-3e5b-4dcc-a95e-58483dec7976 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:49.948952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-522b0be5-57ac-49e4-9306-954313da4344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:49.956034 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6f8d0-3e5b-4dcc-a95e-58483dec7976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:49.964934 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b0be5-57ac-49e4-9306-954313da4344 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:49.985987 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-61a6f8d0-3e5b-4dcc-a95e-58483dec7976 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:49.988092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-61a6f8d0-3e5b-4dcc-a95e-58483dec7976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:50.061366 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46246b68-97c2-4536-9702-c9ee6134727b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:50.062415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46246b68-97c2-4536-9702-c9ee6134727b admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:50.062919 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46246b68-97c2-4536-9702-c9ee6134727b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:50.063431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46246b68-97c2-4536-9702-c9ee6134727b admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:50.064642 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46246b68-97c2-4536-9702-c9ee6134727b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:50.067425 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46246b68-97c2-4536-9702-c9ee6134727b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:50.083316 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1073/2136] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:50.083316 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6f8d0-3e5b-4dcc-a95e-58483dec7976 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:50.083316 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-61a6f8d0-3e5b-4dcc-a95e-58483dec7976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:50.093259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1064/2137] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:50.093259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd2992-7bd2-4450-bacf-55f8eb3f31ed None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:50.093259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd2992-7bd2-4450-bacf-55f8eb3f31ed None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:50.093259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd2992-7bd2-4450-bacf-55f8eb3f31ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:50.101471 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6f8d0-3e5b-4dcc-a95e-58483dec7976 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:50.107508 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99160590-bacf-4beb-8993-af11260fb1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:50.108863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99160590-bacf-4beb-8993-af11260fb1de admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:50.109297 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99160590-bacf-4beb-8993-af11260fb1de admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:50.109749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99160590-bacf-4beb-8993-af11260fb1de admin admin] PATH_INFO: `/v3/projects/ebe5676e1c734d1aa2ca4fb694ac57ed/users/00d2e5a7561d40b29fc62eaec0e2a838/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:50.114578 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d4fd2992-7bd2-4450-bacf-55f8eb3f31ed None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:50.116699 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d4fd2992-7bd2-4450-bacf-55f8eb3f31ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:50.127645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99160590-bacf-4beb-8993-af11260fb1de admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebe5676e1c734d1aa2ca4fb694ac57ed, user_id=00d2e5a7561d40b29fc62eaec0e2a838, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:50.128078 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1065/2138] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:50.128905 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd2992-7bd2-4450-bacf-55f8eb3f31ed None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:50.130367 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99160590-bacf-4beb-8993-af11260fb1de admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:50.140419 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd2992-7bd2-4450-bacf-55f8eb3f31ed None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:50.145893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1074/2139] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:50 2022] PUT /identity/v3/projects/ebe5676e1c734d1aa2ca4fb694ac57ed/users/00d2e5a7561d40b29fc62eaec0e2a838/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:50.146460 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99160590-bacf-4beb-8993-af11260fb1de admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:50.172514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:50.214857 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223b1a3-cd39-480b-9dfb-a2ec31f1b096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:50.216039 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223b1a3-cd39-480b-9dfb-a2ec31f1b096 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:50.216334 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223b1a3-cd39-480b-9dfb-a2ec31f1b096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:50.216619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223b1a3-cd39-480b-9dfb-a2ec31f1b096 admin admin] PATH_INFO: `/v3/projects/8e6befea55844949a070c7899172df7c/users/d91830e475fb4c5fae266ca583d935f9/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:50.225659 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:50.232915 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223b1a3-cd39-480b-9dfb-a2ec31f1b096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e6befea55844949a070c7899172df7c, user_id=d91830e475fb4c5fae266ca583d935f9, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:50.235889 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223b1a3-cd39-480b-9dfb-a2ec31f1b096 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:50.241506 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99160590-bacf-4beb-8993-af11260fb1de admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:50.250588 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1066/2140] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:50 2022] PUT /identity/v3/projects/8e6befea55844949a070c7899172df7c/users/d91830e475fb4c5fae266ca583d935f9/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:50.251128 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d57b09-bbc4-4de6-a898-e1f0efe29099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:50.252025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d57b09-bbc4-4de6-a898-e1f0efe29099 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:50.252131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c62c5-43de-411b-a458-2f9655d7fa56 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:50.252427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c62c5-43de-411b-a458-2f9655d7fa56 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:50.252839 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c62c5-43de-411b-a458-2f9655d7fa56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:50.254036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d57b09-bbc4-4de6-a898-e1f0efe29099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:50.254330 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d57b09-bbc4-4de6-a898-e1f0efe29099 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:50.255335 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d57b09-bbc4-4de6-a898-e1f0efe29099 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:22:50.255655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d57b09-bbc4-4de6-a898-e1f0efe29099 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:50.257280 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d57b09-bbc4-4de6-a898-e1f0efe29099 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:50.275851 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1075/2141] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:22:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:22:50.275851 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d57b09-bbc4-4de6-a898-e1f0efe29099 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:50.276439 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-81d57b09-bbc4-4de6-a898-e1f0efe29099 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:50.291027 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-100c62c5-43de-411b-a458-2f9655d7fa56 None None] MFA Rules not processed for user `00d2e5a7561d40b29fc62eaec0e2a838`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:50.291188 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d57b09-bbc4-4de6-a898-e1f0efe29099 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:50.292282 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-100c62c5-43de-411b-a458-2f9655d7fa56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:50.365037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c121a86-616b-4f41-b01d-36c7ae4b0782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:50.366497 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c121a86-616b-4f41-b01d-36c7ae4b0782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:50.367062 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c121a86-616b-4f41-b01d-36c7ae4b0782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:50.368303 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c121a86-616b-4f41-b01d-36c7ae4b0782 admin admin] PATH_INFO: `/v3/projects/c4bd00a4cfd94739b23da26e08c24ec8` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:50.369652 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c121a86-616b-4f41-b01d-36c7ae4b0782 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4bd00a4cfd94739b23da26e08c24ec8)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:50.372899 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c121a86-616b-4f41-b01d-36c7ae4b0782 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:50.385598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1067/2142] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:50.386902 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c62c5-43de-411b-a458-2f9655d7fa56 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:50.405776 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c62c5-43de-411b-a458-2f9655d7fa56 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:50.420620 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3666d-85b3-4f05-9391-bc39c2cf8cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:50.422099 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3666d-85b3-4f05-9391-bc39c2cf8cae admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:50.423647 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3666d-85b3-4f05-9391-bc39c2cf8cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:50.423647 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3666d-85b3-4f05-9391-bc39c2cf8cae admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:50.423818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3666d-85b3-4f05-9391-bc39c2cf8cae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:50.426001 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3666d-85b3-4f05-9391-bc39c2cf8cae admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:50.442715 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1068/2143] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:50.443680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3666d-85b3-4f05-9391-bc39c2cf8cae admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:50.453031 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0c121a86-616b-4f41-b01d-36c7ae4b0782 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4bd00a4cfd94739b23da26e08c24ec8', 'request_id': 'req-0c121a86-616b-4f41-b01d-36c7ae4b0782'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:50.454504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3666d-85b3-4f05-9391-bc39c2cf8cae admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:50.463212 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0c121a86-616b-4f41-b01d-36c7ae4b0782 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4bd00a4cfd94739b23da26e08c24ec8', 'request_id': 'req-0c121a86-616b-4f41-b01d-36c7ae4b0782'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:50.472409 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0c121a86-616b-4f41-b01d-36c7ae4b0782 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4bd00a4cfd94739b23da26e08c24ec8', 'request_id': 'req-0c121a86-616b-4f41-b01d-36c7ae4b0782'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:50.484166 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0c121a86-616b-4f41-b01d-36c7ae4b0782 admin admin] The token cache is being invalidate because project c4bd00a4cfd94739b23da26e08c24ec8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:50.484470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0c121a86-616b-4f41-b01d-36c7ae4b0782 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:50.485292 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0c121a86-616b-4f41-b01d-36c7ae4b0782 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:50.486903 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1076/2144] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:50 2022] DELETE /identity/v3/projects/c4bd00a4cfd94739b23da26e08c24ec8 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:50.487772 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c121a86-616b-4f41-b01d-36c7ae4b0782 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:50.489127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0c121a86-616b-4f41-b01d-36c7ae4b0782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:50.542939 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6a257-954d-4738-9f2c-7f604801de9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:50.544312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6a257-954d-4738-9f2c-7f604801de9f admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:50.544677 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6a257-954d-4738-9f2c-7f604801de9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:50.544948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6a257-954d-4738-9f2c-7f604801de9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:50.545645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6a257-954d-4738-9f2c-7f604801de9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:50.547587 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6a257-954d-4738-9f2c-7f604801de9f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:50.580046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c121a86-616b-4f41-b01d-36c7ae4b0782 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:50.592907 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1069/2145] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:50 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:50.596568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f4751-c988-4c40-be14-7d6d618bed9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:50.597250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f4751-c988-4c40-be14-7d6d618bed9f service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:50.597532 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f4751-c988-4c40-be14-7d6d618bed9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:50.597806 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f4751-c988-4c40-be14-7d6d618bed9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:50.598814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6a257-954d-4738-9f2c-7f604801de9f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:50.602178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-61a6a257-954d-4738-9f2c-7f604801de9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:50.603790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c48f4751-c988-4c40-be14-7d6d618bed9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:50.619541 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:50.647681 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:50.686593 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f4751-c988-4c40-be14-7d6d618bed9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:50.686768 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6a257-954d-4738-9f2c-7f604801de9f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:50.688512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f4751-c988-4c40-be14-7d6d618bed9f service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:50.691161 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ec06b-d4de-4483-8236-749a0fcbf9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:50.692064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ec06b-d4de-4483-8236-749a0fcbf9f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:50.692358 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ec06b-d4de-4483-8236-749a0fcbf9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:50.692613 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ec06b-d4de-4483-8236-749a0fcbf9f5 admin admin] PATH_INFO: `/v3/projects/8e6befea55844949a070c7899172df7c/users/d91830e475fb4c5fae266ca583d935f9/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:50.695144 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ec06b-d4de-4483-8236-749a0fcbf9f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e6befea55844949a070c7899172df7c, user_id=d91830e475fb4c5fae266ca583d935f9, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:50.697975 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ec06b-d4de-4483-8236-749a0fcbf9f5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:50.701664 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1077/2146] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:50 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:50.702972 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ee99e-06d8-4e5c-a2be-caa713361716 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:50.703290 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ee99e-06d8-4e5c-a2be-caa713361716 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:50.703575 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ee99e-06d8-4e5c-a2be-caa713361716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:50.711650 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1070/2147] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:50 2022] PUT /identity/v3/projects/8e6befea55844949a070c7899172df7c/users/d91830e475fb4c5fae266ca583d935f9/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:50.716189 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ec06b-d4de-4483-8236-749a0fcbf9f5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:50.721426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-378ec06b-d4de-4483-8236-749a0fcbf9f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:50.724542 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-011ee99e-06d8-4e5c-a2be-caa713361716 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:50.725710 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-011ee99e-06d8-4e5c-a2be-caa713361716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:50.785695 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1078/2148] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:50.786046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ec06b-d4de-4483-8236-749a0fcbf9f5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:50.786791 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ee99e-06d8-4e5c-a2be-caa713361716 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:50.788288 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-011ee99e-06d8-4e5c-a2be-caa713361716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:50.790808 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43756b1b-951f-4946-909f-17efe6bbaae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:50.791684 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43756b1b-951f-4946-909f-17efe6bbaae1 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:50.792139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43756b1b-951f-4946-909f-17efe6bbaae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:50.792414 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43756b1b-951f-4946-909f-17efe6bbaae1 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:50.793096 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43756b1b-951f-4946-909f-17efe6bbaae1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:50.795639 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43756b1b-951f-4946-909f-17efe6bbaae1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:50.800268 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ee99e-06d8-4e5c-a2be-caa713361716 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:50.804912 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e9e97-b13f-4a29-b028-71b1ed5c1133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:50.805778 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e9e97-b13f-4a29-b028-71b1ed5c1133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:50.806061 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e9e97-b13f-4a29-b028-71b1ed5c1133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:50.806381 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e9e97-b13f-4a29-b028-71b1ed5c1133 admin admin] PATH_INFO: `/v3/users/00d2e5a7561d40b29fc62eaec0e2a838` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:50.807622 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e9e97-b13f-4a29-b028-71b1ed5c1133 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00d2e5a7561d40b29fc62eaec0e2a838)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:50.810322 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e9e97-b13f-4a29-b028-71b1ed5c1133 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:50.816535 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1071/2149] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:50 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:50.817796 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67772fdb-8423-4af4-bde7-f88604595154 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:50.818088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67772fdb-8423-4af4-bde7-f88604595154 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:50.818363 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67772fdb-8423-4af4-bde7-f88604595154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:50.845796 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-67772fdb-8423-4af4-bde7-f88604595154 None None] MFA Rules not processed for user `d91830e475fb4c5fae266ca583d935f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:50.846996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-67772fdb-8423-4af4-bde7-f88604595154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:50.858571 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-556e9e97-b13f-4a29-b028-71b1ed5c1133 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00d2e5a7561d40b29fc62eaec0e2a838'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:50.874779 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-556e9e97-b13f-4a29-b028-71b1ed5c1133 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00d2e5a7561d40b29fc62eaec0e2a838'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:50.875558 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-556e9e97-b13f-4a29-b028-71b1ed5c1133 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00d2e5a7561d40b29fc62eaec0e2a838'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:50.876207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-556e9e97-b13f-4a29-b028-71b1ed5c1133 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00d2e5a7561d40b29fc62eaec0e2a838'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:50.887068 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-556e9e97-b13f-4a29-b028-71b1ed5c1133 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00d2e5a7561d40b29fc62eaec0e2a838'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:50.907086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1079/2150] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:50 2022] DELETE /identity/v3/users/00d2e5a7561d40b29fc62eaec0e2a838 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:50.908569 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb71523-80df-4436-b569-98d82e84b943 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:50.908929 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb71523-80df-4436-b569-98d82e84b943 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:50.909296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb71523-80df-4436-b569-98d82e84b943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:50.935269 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1072/2151] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:50 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:50.935269 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67772fdb-8423-4af4-bde7-f88604595154 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:50.935269 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-67772fdb-8423-4af4-bde7-f88604595154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:50.935747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-4eb71523-80df-4436-b569-98d82e84b943 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:50.935747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4eb71523-80df-4436-b569-98d82e84b943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:50.944478 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67772fdb-8423-4af4-bde7-f88604595154 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:50.958421 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:50.974633 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:50.999540 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1080/2152] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:51.000905 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf3b3a-376d-45e6-98d4-e0fb17590a02 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:51.001197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf3b3a-376d-45e6-98d4-e0fb17590a02 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:51.001473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf3b3a-376d-45e6-98d4-e0fb17590a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:51.006929 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7164698e-05ca-4df1-84da-7a636b1bf4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:51.007659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164698e-05ca-4df1-84da-7a636b1bf4ba admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:51.007928 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164698e-05ca-4df1-84da-7a636b1bf4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:51.008206 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164698e-05ca-4df1-84da-7a636b1bf4ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:51.008946 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7164698e-05ca-4df1-84da-7a636b1bf4ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:51.010550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7164698e-05ca-4df1-84da-7a636b1bf4ba admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:51.019958 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-64cf3b3a-376d-45e6-98d4-e0fb17590a02 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:51.021163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-64cf3b3a-376d-45e6-98d4-e0fb17590a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:51.024469 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1073/2153] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:51.024469 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7164698e-05ca-4df1-84da-7a636b1bf4ba admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:51.024469 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7164698e-05ca-4df1-84da-7a636b1bf4ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:51.029245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1081/2154] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:51.030037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf3b3a-376d-45e6-98d4-e0fb17590a02 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:51.044998 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf3b3a-376d-45e6-98d4-e0fb17590a02 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:51.049610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85355f72-565f-452b-bdd0-91f16146dc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:51.050264 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85355f72-565f-452b-bdd0-91f16146dc7b admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:51.050547 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85355f72-565f-452b-bdd0-91f16146dc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:51.050821 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85355f72-565f-452b-bdd0-91f16146dc7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:51.051579 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85355f72-565f-452b-bdd0-91f16146dc7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:22:51.051887 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85355f72-565f-452b-bdd0-91f16146dc7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:51.053472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85355f72-565f-452b-bdd0-91f16146dc7b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:51.066070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1082/2155] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:22:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:22:51.066876 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85355f72-565f-452b-bdd0-91f16146dc7b admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:51.077131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85355f72-565f-452b-bdd0-91f16146dc7b admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:51.081748 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b1981-9612-4821-98a9-e0867a558f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:51.082340 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7164698e-05ca-4df1-84da-7a636b1bf4ba admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:51.082815 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b1981-9612-4821-98a9-e0867a558f26 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:51.082815 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b1981-9612-4821-98a9-e0867a558f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:51.082968 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b1981-9612-4821-98a9-e0867a558f26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:51.083831 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b1981-9612-4821-98a9-e0867a558f26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:51.085390 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b1981-9612-4821-98a9-e0867a558f26 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:51.086286 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2553d-8181-4eeb-8333-c25ecb0f76d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:51.086934 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2553d-8181-4eeb-8333-c25ecb0f76d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:51.087251 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2553d-8181-4eeb-8333-c25ecb0f76d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:51.087531 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2553d-8181-4eeb-8333-c25ecb0f76d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:51.100729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1083/2156] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:51.100729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b1981-9612-4821-98a9-e0867a558f26 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:51.100729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b94b1981-9612-4821-98a9-e0867a558f26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:51.156112 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b1981-9612-4821-98a9-e0867a558f26 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:51.157791 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff2553d-8181-4eeb-8333-c25ecb0f76d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:51.159283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff2553d-8181-4eeb-8333-c25ecb0f76d9 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:51.161807 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11010d7c-32da-4e66-8133-f057e4f18c87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:51.162467 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11010d7c-32da-4e66-8133-f057e4f18c87 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:51.162752 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11010d7c-32da-4e66-8133-f057e4f18c87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:51.163027 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11010d7c-32da-4e66-8133-f057e4f18c87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:51.169727 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1074/2157] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:51 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:51.170906 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2553d-8181-4eeb-8333-c25ecb0f76d9 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:51.173011 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11010d7c-32da-4e66-8133-f057e4f18c87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:51.174481 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11010d7c-32da-4e66-8133-f057e4f18c87 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:51.181259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2553d-8181-4eeb-8333-c25ecb0f76d9 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:51.185057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1084/2158] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:51.186035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11010d7c-32da-4e66-8133-f057e4f18c87 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:51.186854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d839ee-6f0c-4395-bfd1-25f21adefa95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:51.187694 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d839ee-6f0c-4395-bfd1-25f21adefa95 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:51.187956 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d839ee-6f0c-4395-bfd1-25f21adefa95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:51.188384 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d839ee-6f0c-4395-bfd1-25f21adefa95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:51.189045 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d839ee-6f0c-4395-bfd1-25f21adefa95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:51.191461 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d839ee-6f0c-4395-bfd1-25f21adefa95 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:51.197383 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11010d7c-32da-4e66-8133-f057e4f18c87 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:51.203853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ef333-7ab2-4da7-8621-e38bdc813b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:51.204714 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ef333-7ab2-4da7-8621-e38bdc813b3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:51.205004 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ef333-7ab2-4da7-8621-e38bdc813b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:51.205863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ef333-7ab2-4da7-8621-e38bdc813b3b admin admin] PATH_INFO: `/v3/projects/8e09c0fb94114711acd6f6b0c2148ec3/users/5809fb226baa4664a703ac5c7345cb80/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:51.223027 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266ef333-7ab2-4da7-8621-e38bdc813b3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e09c0fb94114711acd6f6b0c2148ec3, user_id=5809fb226baa4664a703ac5c7345cb80, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:51.226152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266ef333-7ab2-4da7-8621-e38bdc813b3b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:51.230430 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1075/2159] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:51 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:51.235119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d839ee-6f0c-4395-bfd1-25f21adefa95 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:51.243264 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1085/2160] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:51 2022] PUT /identity/v3/projects/8e09c0fb94114711acd6f6b0c2148ec3/users/5809fb226baa4664a703ac5c7345cb80/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:51.259691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f310af2-6917-4efc-b445-df71684ffb60 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:51.259982 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f310af2-6917-4efc-b445-df71684ffb60 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:51.260266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f310af2-6917-4efc-b445-df71684ffb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:51.291392 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6f310af2-6917-4efc-b445-df71684ffb60 None None] MFA Rules not processed for user `5809fb226baa4664a703ac5c7345cb80`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:51.292806 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6f310af2-6917-4efc-b445-df71684ffb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:51.305423 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:51.329025 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d839ee-6f0c-4395-bfd1-25f21adefa95 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:51.384127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1086/2161] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:51.388064 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f310af2-6917-4efc-b445-df71684ffb60 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:51.397122 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb976c-a1f4-459c-bbea-aa4b425bfbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:51.397806 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb976c-a1f4-459c-bbea-aa4b425bfbd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:51.398096 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb976c-a1f4-459c-bbea-aa4b425bfbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:51.398375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb976c-a1f4-459c-bbea-aa4b425bfbd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:51.399175 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eb976c-a1f4-459c-bbea-aa4b425bfbd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:51.402120 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eb976c-a1f4-459c-bbea-aa4b425bfbd5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:51.409690 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:51.428319 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1076/2162] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:51 2022] POST /identity/v3/users => generated 288 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:51.429652 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb976c-a1f4-459c-bbea-aa4b425bfbd5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:51.440736 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb976c-a1f4-459c-bbea-aa4b425bfbd5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:51.446931 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b1c2dc-e478-4048-ba7a-361ecb77ee5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:51.447916 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1c2dc-e478-4048-ba7a-361ecb77ee5f admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:51.448196 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f310af2-6917-4efc-b445-df71684ffb60 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:51.448555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1c2dc-e478-4048-ba7a-361ecb77ee5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:51.448993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1c2dc-e478-4048-ba7a-361ecb77ee5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:51.450476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b1c2dc-e478-4048-ba7a-361ecb77ee5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:51.452870 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63d533-5ebc-4e0c-92ed-f74c062aca96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:51.453142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b1c2dc-e478-4048-ba7a-361ecb77ee5f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:51.453520 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63d533-5ebc-4e0c-92ed-f74c062aca96 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:51.453805 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63d533-5ebc-4e0c-92ed-f74c062aca96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:51.454083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63d533-5ebc-4e0c-92ed-f74c062aca96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:51.456192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-aa63d533-5ebc-4e0c-92ed-f74c062aca96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:51.486326 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1077/2163] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:51 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:51.486326 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b1c2dc-e478-4048-ba7a-361ecb77ee5f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:51.525266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa63d533-5ebc-4e0c-92ed-f74c062aca96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:51.526823 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa63d533-5ebc-4e0c-92ed-f74c062aca96 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:51.537440 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1087/2164] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:51 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:51.538488 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63d533-5ebc-4e0c-92ed-f74c062aca96 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:51.550287 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63d533-5ebc-4e0c-92ed-f74c062aca96 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:51.564570 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b1c2dc-e478-4048-ba7a-361ecb77ee5f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:51.569604 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a78bc-3e48-4737-935e-a330bbd93f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:51.570277 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a78bc-3e48-4737-935e-a330bbd93f7f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:51.570585 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a78bc-3e48-4737-935e-a330bbd93f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:51.570859 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a78bc-3e48-4737-935e-a330bbd93f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:51.571625 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a78bc-3e48-4737-935e-a330bbd93f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:51.573212 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a78bc-3e48-4737-935e-a330bbd93f7f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:51.587719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1078/2165] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:51.587719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a78bc-3e48-4737-935e-a330bbd93f7f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:51.589110 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-760a78bc-3e48-4737-935e-a330bbd93f7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:51.605959 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a78bc-3e48-4737-935e-a330bbd93f7f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:51.613103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d75ce4-7e77-4e18-ac1d-a784a4efec2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:51.613969 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d75ce4-7e77-4e18-ac1d-a784a4efec2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:51.614248 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d75ce4-7e77-4e18-ac1d-a784a4efec2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:51.614525 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d75ce4-7e77-4e18-ac1d-a784a4efec2c admin admin] PATH_INFO: `/v3/projects/ebe5676e1c734d1aa2ca4fb694ac57ed` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:51.615793 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d75ce4-7e77-4e18-ac1d-a784a4efec2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebe5676e1c734d1aa2ca4fb694ac57ed)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:51.617730 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d75ce4-7e77-4e18-ac1d-a784a4efec2c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:51.619502 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735b85a-6cea-4ad0-b8a8-d6956b23ce6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:51.620303 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735b85a-6cea-4ad0-b8a8-d6956b23ce6d admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:51.620597 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735b85a-6cea-4ad0-b8a8-d6956b23ce6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:51.620879 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735b85a-6cea-4ad0-b8a8-d6956b23ce6d admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:51.621582 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735b85a-6cea-4ad0-b8a8-d6956b23ce6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:51.624410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735b85a-6cea-4ad0-b8a8-d6956b23ce6d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:51.650958 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1088/2166] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:51 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:51.651800 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735b85a-6cea-4ad0-b8a8-d6956b23ce6d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:51.665819 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735b85a-6cea-4ad0-b8a8-d6956b23ce6d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:51.668777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:51.685312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-70d75ce4-7e77-4e18-ac1d-a784a4efec2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebe5676e1c734d1aa2ca4fb694ac57ed', 'request_id': 'req-70d75ce4-7e77-4e18-ac1d-a784a4efec2c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:51.698330 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-70d75ce4-7e77-4e18-ac1d-a784a4efec2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebe5676e1c734d1aa2ca4fb694ac57ed', 'request_id': 'req-70d75ce4-7e77-4e18-ac1d-a784a4efec2c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:51.707528 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-70d75ce4-7e77-4e18-ac1d-a784a4efec2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebe5676e1c734d1aa2ca4fb694ac57ed', 'request_id': 'req-70d75ce4-7e77-4e18-ac1d-a784a4efec2c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:51.730024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-70d75ce4-7e77-4e18-ac1d-a784a4efec2c admin admin] The token cache is being invalidate because project ebe5676e1c734d1aa2ca4fb694ac57ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:51.730343 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-70d75ce4-7e77-4e18-ac1d-a784a4efec2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:51.732830 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-70d75ce4-7e77-4e18-ac1d-a784a4efec2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:51.734753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1079/2167] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:51 2022] DELETE /identity/v3/projects/ebe5676e1c734d1aa2ca4fb694ac57ed => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:51.735847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d75ce4-7e77-4e18-ac1d-a784a4efec2c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:51.737493 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-70d75ce4-7e77-4e18-ac1d-a784a4efec2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:51.749269 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47726c9a-b8f4-4f67-9df0-392f57917382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:51.749951 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47726c9a-b8f4-4f67-9df0-392f57917382 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:51.750359 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47726c9a-b8f4-4f67-9df0-392f57917382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:51.750580 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47726c9a-b8f4-4f67-9df0-392f57917382 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:51.751454 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47726c9a-b8f4-4f67-9df0-392f57917382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:51.753519 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47726c9a-b8f4-4f67-9df0-392f57917382 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:51.760128 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:51.766539 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1089/2168] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:51.768200 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae9348-dfdb-4f91-997f-6747bdf217e0 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:51.769168 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae9348-dfdb-4f91-997f-6747bdf217e0 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:51.770137 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae9348-dfdb-4f91-997f-6747bdf217e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:51.793088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-51ae9348-dfdb-4f91-997f-6747bdf217e0 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:51.795051 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-51ae9348-dfdb-4f91-997f-6747bdf217e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:51.806029 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1090/2169] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:51.807193 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae9348-dfdb-4f91-997f-6747bdf217e0 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:51.808595 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-51ae9348-dfdb-4f91-997f-6747bdf217e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:51.822209 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae9348-dfdb-4f91-997f-6747bdf217e0 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:51.828369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19eaa8c-f1b5-452c-bfd3-6aaac1f53908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:51.829337 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19eaa8c-f1b5-452c-bfd3-6aaac1f53908 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:51.829627 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19eaa8c-f1b5-452c-bfd3-6aaac1f53908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:51.829932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19eaa8c-f1b5-452c-bfd3-6aaac1f53908 admin admin] PATH_INFO: `/v3/projects/1cd5dfddd24c42779f9950fc18957ac5/users/4df6444245864bfba9093644528f77ec/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:51.833040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d75ce4-7e77-4e18-ac1d-a784a4efec2c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:51.838604 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8950b-6f9c-426c-b62a-e4c4bc8bfbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:51.839463 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8950b-6f9c-426c-b62a-e4c4bc8bfbf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:51.839770 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8950b-6f9c-426c-b62a-e4c4bc8bfbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:51.840424 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8950b-6f9c-426c-b62a-e4c4bc8bfbf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:51.841596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8950b-6f9c-426c-b62a-e4c4bc8bfbf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:51.843202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8950b-6f9c-426c-b62a-e4c4bc8bfbf2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:51.847181 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19eaa8c-f1b5-452c-bfd3-6aaac1f53908 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cd5dfddd24c42779f9950fc18957ac5, user_id=4df6444245864bfba9093644528f77ec, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:51.850251 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19eaa8c-f1b5-452c-bfd3-6aaac1f53908 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:51.855573 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1080/2170] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:51.857445 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ece2ab-7439-46bf-8328-77dc25f7bf66 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:51.857730 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ece2ab-7439-46bf-8328-77dc25f7bf66 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:51.858002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ece2ab-7439-46bf-8328-77dc25f7bf66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:51.879352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1091/2171] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:51 2022] PUT /identity/v3/projects/1cd5dfddd24c42779f9950fc18957ac5/users/4df6444245864bfba9093644528f77ec/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:51.879352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19eaa8c-f1b5-452c-bfd3-6aaac1f53908 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:51.879954 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b8ece2ab-7439-46bf-8328-77dc25f7bf66 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:51.881472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b8ece2ab-7439-46bf-8328-77dc25f7bf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:51.984812 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19eaa8c-f1b5-452c-bfd3-6aaac1f53908 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:51.985806 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1081/2172] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:51.987355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fae8b-a5db-466b-8157-f9278727850e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:51.987812 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fae8b-a5db-466b-8157-f9278727850e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:51.988277 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fae8b-a5db-466b-8157-f9278727850e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:51.993817 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91c775-cc49-49ec-b9fa-be46b4ee2ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:51.994999 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91c775-cc49-49ec-b9fa-be46b4ee2ac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:51.995314 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91c775-cc49-49ec-b9fa-be46b4ee2ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:51.995694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91c775-cc49-49ec-b9fa-be46b4ee2ac8 admin admin] PATH_INFO: `/v3/projects/6414ec0a008f473881894cdeaa7aabe8/users/ebaa488d45374133aee1bad1d6b091e7/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:52.017094 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91c775-cc49-49ec-b9fa-be46b4ee2ac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6414ec0a008f473881894cdeaa7aabe8, user_id=ebaa488d45374133aee1bad1d6b091e7, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:52.019816 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91c775-cc49-49ec-b9fa-be46b4ee2ac8 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:52.020089 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-747fae8b-a5db-466b-8157-f9278727850e None None] MFA Rules not processed for user `4df6444245864bfba9093644528f77ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:52.021350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-747fae8b-a5db-466b-8157-f9278727850e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:52.034407 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1092/2173] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:51 2022] PUT /identity/v3/projects/6414ec0a008f473881894cdeaa7aabe8/users/ebaa488d45374133aee1bad1d6b091e7/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:52.035554 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91c775-cc49-49ec-b9fa-be46b4ee2ac8 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:52.089868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1082/2174] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:52.091398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fae8b-a5db-466b-8157-f9278727850e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:52.097501 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:52.101852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91c775-cc49-49ec-b9fa-be46b4ee2ac8 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:52.103169 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fae8b-a5db-466b-8157-f9278727850e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:52.107381 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808eff6-2a54-428f-aece-cb65eef1acd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:52.108067 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808eff6-2a54-428f-aece-cb65eef1acd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:52.108356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808eff6-2a54-428f-aece-cb65eef1acd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:52.108630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808eff6-2a54-428f-aece-cb65eef1acd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:52.109574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6808eff6-2a54-428f-aece-cb65eef1acd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:22:52.109695 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6808eff6-2a54-428f-aece-cb65eef1acd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:52.109990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3250261-1bf6-42a8-aa50-737077e89e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:52.110683 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3250261-1bf6-42a8-aa50-737077e89e08 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:52.111094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3250261-1bf6-42a8-aa50-737077e89e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:52.111351 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3250261-1bf6-42a8-aa50-737077e89e08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:52.111454 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6808eff6-2a54-428f-aece-cb65eef1acd4 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:52.112351 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3250261-1bf6-42a8-aa50-737077e89e08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:52.114443 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3250261-1bf6-42a8-aa50-737077e89e08 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:52.123575 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1093/2175] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:22:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:22:52.124366 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808eff6-2a54-428f-aece-cb65eef1acd4 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:52.125018 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1083/2176] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:52.128038 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3250261-1bf6-42a8-aa50-737077e89e08 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:52.138304 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808eff6-2a54-428f-aece-cb65eef1acd4 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:52.138448 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3250261-1bf6-42a8-aa50-737077e89e08 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:52.142841 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb688f3f-f5e9-481f-83a7-929f6ea70b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:52.143063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d8085-1687-4e26-b24f-58f5eb2ddb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:52.143536 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb688f3f-f5e9-481f-83a7-929f6ea70b07 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:52.143736 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d8085-1687-4e26-b24f-58f5eb2ddb29 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:52.143840 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb688f3f-f5e9-481f-83a7-929f6ea70b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:52.144019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d8085-1687-4e26-b24f-58f5eb2ddb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:52.144133 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb688f3f-f5e9-481f-83a7-929f6ea70b07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:52.144294 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d8085-1687-4e26-b24f-58f5eb2ddb29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:52.144763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb688f3f-f5e9-481f-83a7-929f6ea70b07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:52.144977 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d8085-1687-4e26-b24f-58f5eb2ddb29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:52.146689 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb688f3f-f5e9-481f-83a7-929f6ea70b07 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:52.146876 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d8085-1687-4e26-b24f-58f5eb2ddb29 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:52.175535 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1094/2177] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:52 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:52.176337 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb688f3f-f5e9-481f-83a7-929f6ea70b07 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:52.184143 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1084/2178] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:52 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:52.184143 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d8085-1687-4e26-b24f-58f5eb2ddb29 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:52.204028 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:52.240324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb688f3f-f5e9-481f-83a7-929f6ea70b07 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:52.245207 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d8085-1687-4e26-b24f-58f5eb2ddb29 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:52.245334 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6429a-9cf7-4301-a2a6-b9d8b87c0109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:52.246135 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6429a-9cf7-4301-a2a6-b9d8b87c0109 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:52.246422 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6429a-9cf7-4301-a2a6-b9d8b87c0109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:52.246856 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6429a-9cf7-4301-a2a6-b9d8b87c0109 admin admin] PATH_INFO: `/v3/projects/6414ec0a008f473881894cdeaa7aabe8/users/ebaa488d45374133aee1bad1d6b091e7/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:52.249002 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6429a-9cf7-4301-a2a6-b9d8b87c0109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6414ec0a008f473881894cdeaa7aabe8, user_id=ebaa488d45374133aee1bad1d6b091e7, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:52.249895 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d22292-ecdd-4a9d-8335-256af86acb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:52.250559 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d22292-ecdd-4a9d-8335-256af86acb2e admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:52.250838 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d22292-ecdd-4a9d-8335-256af86acb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:52.251112 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d22292-ecdd-4a9d-8335-256af86acb2e admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:52.251712 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6429a-9cf7-4301-a2a6-b9d8b87c0109 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:52.251853 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d22292-ecdd-4a9d-8335-256af86acb2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:52.253745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d22292-ecdd-4a9d-8335-256af86acb2e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:52.262926 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1095/2179] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:52 2022] PUT /identity/v3/projects/6414ec0a008f473881894cdeaa7aabe8/users/ebaa488d45374133aee1bad1d6b091e7/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:52.263931 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6429a-9cf7-4301-a2a6-b9d8b87c0109 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:52.277241 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1085/2180] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:52 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:52.279132 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f1a97-c0ed-44e3-bd11-fd541e13ebbb None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:52.279446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f1a97-c0ed-44e3-bd11-fd541e13ebbb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:52.279721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f1a97-c0ed-44e3-bd11-fd541e13ebbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:52.305416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-767f1a97-c0ed-44e3-bd11-fd541e13ebbb None None] MFA Rules not processed for user `ebaa488d45374133aee1bad1d6b091e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:52.306619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-767f1a97-c0ed-44e3-bd11-fd541e13ebbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:52.337307 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6429a-9cf7-4301-a2a6-b9d8b87c0109 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:52.342660 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ba857-1d6c-4664-8df0-b51803755d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:52.343354 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ba857-1d6c-4664-8df0-b51803755d6d admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:52.343637 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ba857-1d6c-4664-8df0-b51803755d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:52.343912 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ba857-1d6c-4664-8df0-b51803755d6d admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:52.344596 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ba857-1d6c-4664-8df0-b51803755d6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:52.346505 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ba857-1d6c-4664-8df0-b51803755d6d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:52.367123 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1096/2181] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:52 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:52.367969 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ba857-1d6c-4664-8df0-b51803755d6d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:52.377945 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1086/2182] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:52 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:52.378321 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ba857-1d6c-4664-8df0-b51803755d6d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:52.378844 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f1a97-c0ed-44e3-bd11-fd541e13ebbb None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:52.385499 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e50809-d0a7-4446-aeb2-41c921ada5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:52.386175 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e50809-d0a7-4446-aeb2-41c921ada5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:52.386452 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e50809-d0a7-4446-aeb2-41c921ada5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:52.386784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e50809-d0a7-4446-aeb2-41c921ada5a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:52.388112 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e50809-d0a7-4446-aeb2-41c921ada5a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:52.389966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e50809-d0a7-4446-aeb2-41c921ada5a4 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:52.390757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f1a97-c0ed-44e3-bd11-fd541e13ebbb None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:52.395420 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9859a-8279-4188-a8db-018cca12c0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:52.396076 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9859a-8279-4188-a8db-018cca12c0f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:52.396358 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9859a-8279-4188-a8db-018cca12c0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:52.396629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9859a-8279-4188-a8db-018cca12c0f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:52.397345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c9859a-8279-4188-a8db-018cca12c0f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:52.398906 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c9859a-8279-4188-a8db-018cca12c0f5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:52.400079 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1097/2183] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:52.400863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e50809-d0a7-4446-aeb2-41c921ada5a4 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:52.402486 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b3e50809-d0a7-4446-aeb2-41c921ada5a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:52.410987 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1087/2184] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:52.411923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9859a-8279-4188-a8db-018cca12c0f5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:52.422713 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9859a-8279-4188-a8db-018cca12c0f5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:52.427469 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026f9a3-573b-45bc-b6f5-1b7d8c6b440a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:52.428342 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026f9a3-573b-45bc-b6f5-1b7d8c6b440a admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:52.428621 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026f9a3-573b-45bc-b6f5-1b7d8c6b440a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:52.428895 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026f9a3-573b-45bc-b6f5-1b7d8c6b440a admin admin] PATH_INFO: `/v3/projects/4e33eaf5dc5148bab6abd57a63b07018/users/a3049952da084732b189ab26281f2a02/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:52.444656 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026f9a3-573b-45bc-b6f5-1b7d8c6b440a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e33eaf5dc5148bab6abd57a63b07018, user_id=a3049952da084732b189ab26281f2a02, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:52.447331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026f9a3-573b-45bc-b6f5-1b7d8c6b440a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:52.460660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1088/2185] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:52 2022] PUT /identity/v3/projects/4e33eaf5dc5148bab6abd57a63b07018/users/a3049952da084732b189ab26281f2a02/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:52.461403 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026f9a3-573b-45bc-b6f5-1b7d8c6b440a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:52.464079 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e50809-d0a7-4446-aeb2-41c921ada5a4 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:52.471714 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:52.517438 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f42b9-ed1b-4ff7-b25c-f7e7f3b167e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:52.518094 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f42b9-ed1b-4ff7-b25c-f7e7f3b167e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:52.518385 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f42b9-ed1b-4ff7-b25c-f7e7f3b167e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:52.518656 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f42b9-ed1b-4ff7-b25c-f7e7f3b167e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:52.523854 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:52.526936 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026f9a3-573b-45bc-b6f5-1b7d8c6b440a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:52.533019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208be2d-c4cd-4380-804a-52c9ff90bf0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:52.533697 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208be2d-c4cd-4380-804a-52c9ff90bf0f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:52.533979 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208be2d-c4cd-4380-804a-52c9ff90bf0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:52.534250 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208be2d-c4cd-4380-804a-52c9ff90bf0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:52.534977 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e208be2d-c4cd-4380-804a-52c9ff90bf0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:52.536610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e208be2d-c4cd-4380-804a-52c9ff90bf0f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:52.547262 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1089/2186] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:52.548065 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208be2d-c4cd-4380-804a-52c9ff90bf0f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:52.557874 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208be2d-c4cd-4380-804a-52c9ff90bf0f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:52.562675 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed9f01-3455-4525-a11c-de49aaf6d9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:52.563340 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed9f01-3455-4525-a11c-de49aaf6d9e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:52.563624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed9f01-3455-4525-a11c-de49aaf6d9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:52.564001 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed9f01-3455-4525-a11c-de49aaf6d9e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:52.564716 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed9f01-3455-4525-a11c-de49aaf6d9e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:52.566281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed9f01-3455-4525-a11c-de49aaf6d9e7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:52.578315 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1090/2187] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:52.578315 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed9f01-3455-4525-a11c-de49aaf6d9e7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:52.584881 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f42b9-ed1b-4ff7-b25c-f7e7f3b167e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:52.586344 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f42b9-ed1b-4ff7-b25c-f7e7f3b167e2 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:52.591719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed9f01-3455-4525-a11c-de49aaf6d9e7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:52.596838 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630ccda-b32c-4a83-85dd-70d67a9c9cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:52.597974 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630ccda-b32c-4a83-85dd-70d67a9c9cd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:52.598086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1098/2188] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:52 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:52.598207 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630ccda-b32c-4a83-85dd-70d67a9c9cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:52.598342 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630ccda-b32c-4a83-85dd-70d67a9c9cd8 admin admin] PATH_INFO: `/v3/projects/a087eb5b34f04e51ad8462cc4deab158/users/3d6162c3ea6b4ee196fe2266bddc7044/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:52.598577 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f42b9-ed1b-4ff7-b25c-f7e7f3b167e2 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:52.609993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f42b9-ed1b-4ff7-b25c-f7e7f3b167e2 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:52.615393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23a36d-5a40-42c0-984c-81efe52e0bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:52.616276 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23a36d-5a40-42c0-984c-81efe52e0bcd admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:52.616556 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23a36d-5a40-42c0-984c-81efe52e0bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:52.616875 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0630ccda-b32c-4a83-85dd-70d67a9c9cd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a087eb5b34f04e51ad8462cc4deab158, user_id=3d6162c3ea6b4ee196fe2266bddc7044, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:52.617070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23a36d-5a40-42c0-984c-81efe52e0bcd admin admin] PATH_INFO: `/v3/projects/4e33eaf5dc5148bab6abd57a63b07018/users/a3049952da084732b189ab26281f2a02/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:52.619049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23a36d-5a40-42c0-984c-81efe52e0bcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e33eaf5dc5148bab6abd57a63b07018, user_id=a3049952da084732b189ab26281f2a02, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:52.619472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0630ccda-b32c-4a83-85dd-70d67a9c9cd8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:52.621749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23a36d-5a40-42c0-984c-81efe52e0bcd admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:52.634195 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1099/2189] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:52 2022] PUT /identity/v3/projects/4e33eaf5dc5148bab6abd57a63b07018/users/a3049952da084732b189ab26281f2a02/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:52.634912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1091/2190] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:52 2022] PUT /identity/v3/projects/a087eb5b34f04e51ad8462cc4deab158/users/3d6162c3ea6b4ee196fe2266bddc7044/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:52.645803 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037388a0-3b41-456a-9c61-6793030288c6 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:52.646070 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e888bfe-c810-47d6-a293-0d9aa5e3356f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:52.646198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037388a0-3b41-456a-9c61-6793030288c6 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:52.646400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037388a0-3b41-456a-9c61-6793030288c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:52.646508 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e888bfe-c810-47d6-a293-0d9aa5e3356f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:52.646672 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e888bfe-c810-47d6-a293-0d9aa5e3356f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:52.675813 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-037388a0-3b41-456a-9c61-6793030288c6 None None] MFA Rules not processed for user `a3049952da084732b189ab26281f2a02`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:52.676371 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4e888bfe-c810-47d6-a293-0d9aa5e3356f None None] MFA Rules not processed for user `3d6162c3ea6b4ee196fe2266bddc7044`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:52.677088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-037388a0-3b41-456a-9c61-6793030288c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:52.677832 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4e888bfe-c810-47d6-a293-0d9aa5e3356f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:52.747851 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1100/2191] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:52 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:52.748701 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037388a0-3b41-456a-9c61-6793030288c6 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:52.749139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1092/2192] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:52.753560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e888bfe-c810-47d6-a293-0d9aa5e3356f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:52.803585 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037388a0-3b41-456a-9c61-6793030288c6 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:52.807328 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b45fb7-e824-4b37-bb3f-740a281e41cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:52.807977 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b45fb7-e824-4b37-bb3f-740a281e41cd service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:52.808262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b45fb7-e824-4b37-bb3f-740a281e41cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:52.808540 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b45fb7-e824-4b37-bb3f-740a281e41cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:52.811194 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-67b45fb7-e824-4b37-bb3f-740a281e41cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:52.814070 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e888bfe-c810-47d6-a293-0d9aa5e3356f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:52.818594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76efdb-8d2c-4948-b188-7a055f66b95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:52.819271 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76efdb-8d2c-4948-b188-7a055f66b95f admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:52.819555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76efdb-8d2c-4948-b188-7a055f66b95f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:52.819829 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76efdb-8d2c-4948-b188-7a055f66b95f admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:52.820521 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a76efdb-8d2c-4948-b188-7a055f66b95f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:52.822401 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a76efdb-8d2c-4948-b188-7a055f66b95f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:52.826409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:52.844606 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:52.852069 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1093/2193] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:52 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:52.856303 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76efdb-8d2c-4948-b188-7a055f66b95f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:52.857984 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2a76efdb-8d2c-4948-b188-7a055f66b95f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:52.876173 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b45fb7-e824-4b37-bb3f-740a281e41cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:52.877614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b45fb7-e824-4b37-bb3f-740a281e41cd service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:52.888046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1101/2194] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:52.888849 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b45fb7-e824-4b37-bb3f-740a281e41cd service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:52.898910 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b45fb7-e824-4b37-bb3f-740a281e41cd service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:52.930671 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76efdb-8d2c-4948-b188-7a055f66b95f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:52.934847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec789f68-b14b-4b47-a310-84d8be1d361a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:52.935709 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec789f68-b14b-4b47-a310-84d8be1d361a service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:52.935991 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec789f68-b14b-4b47-a310-84d8be1d361a service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:52.936264 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec789f68-b14b-4b47-a310-84d8be1d361a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:52.999774 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2b93b-b04c-4b73-8619-46e2d7d59a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:53.000844 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2b93b-b04c-4b73-8619-46e2d7d59a7f admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:53.002441 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2b93b-b04c-4b73-8619-46e2d7d59a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:53.002904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2b93b-b04c-4b73-8619-46e2d7d59a7f admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:53.004220 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2b93b-b04c-4b73-8619-46e2d7d59a7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:53.008299 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2b93b-b04c-4b73-8619-46e2d7d59a7f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:53.026334 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec789f68-b14b-4b47-a310-84d8be1d361a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:53.029000 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec789f68-b14b-4b47-a310-84d8be1d361a service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:53.056763 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1094/2195] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:53.078758 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1102/2196] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:52 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:53.080436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec789f68-b14b-4b47-a310-84d8be1d361a service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:53.102482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec789f68-b14b-4b47-a310-84d8be1d361a service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:53.115750 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a6f65-dccf-4fb0-90cb-2f0a03541c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:53.116914 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a6f65-dccf-4fb0-90cb-2f0a03541c15 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:53.117500 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a6f65-dccf-4fb0-90cb-2f0a03541c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:53.118002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a6f65-dccf-4fb0-90cb-2f0a03541c15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:53.119530 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3a6f65-dccf-4fb0-90cb-2f0a03541c15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:53.127310 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3a6f65-dccf-4fb0-90cb-2f0a03541c15 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:53.147305 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1095/2197] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:53 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:53.149913 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2b93b-b04c-4b73-8619-46e2d7d59a7f admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:53.161527 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2b93b-b04c-4b73-8619-46e2d7d59a7f admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:53.168392 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871db4b-955b-43d7-9ee5-759730281093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:53.169559 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871db4b-955b-43d7-9ee5-759730281093 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:53.170280 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871db4b-955b-43d7-9ee5-759730281093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:53.170676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871db4b-955b-43d7-9ee5-759730281093 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:53.171679 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5871db4b-955b-43d7-9ee5-759730281093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:53.173480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5871db4b-955b-43d7-9ee5-759730281093 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:53.186941 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1103/2198] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:22:53.191142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a6f65-dccf-4fb0-90cb-2f0a03541c15 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:53.203838 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a6f65-dccf-4fb0-90cb-2f0a03541c15 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:53.211699 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68561979-e387-4e95-9cd1-b9991f3486e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:53.211699 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68561979-e387-4e95-9cd1-b9991f3486e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:53.211699 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68561979-e387-4e95-9cd1-b9991f3486e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:53.211699 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68561979-e387-4e95-9cd1-b9991f3486e7 admin admin] PATH_INFO: `/v3/projects/5afcaee98c1341cba05a9ac50a0bc4ec/users/8f17057c890d448290fbd1ed76205351/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:53.231014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68561979-e387-4e95-9cd1-b9991f3486e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5afcaee98c1341cba05a9ac50a0bc4ec, user_id=8f17057c890d448290fbd1ed76205351, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:53.233935 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68561979-e387-4e95-9cd1-b9991f3486e7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:53.249097 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1096/2199] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:53 2022] PUT /identity/v3/projects/5afcaee98c1341cba05a9ac50a0bc4ec/users/8f17057c890d448290fbd1ed76205351/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:22:53.259144 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4101e63-736b-4a05-bf77-6a97bb131c6b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:53.259467 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4101e63-736b-4a05-bf77-6a97bb131c6b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:53.259752 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4101e63-736b-4a05-bf77-6a97bb131c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:53.296158 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-e4101e63-736b-4a05-bf77-6a97bb131c6b None None] MFA Rules not processed for user `8f17057c890d448290fbd1ed76205351`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:53.297459 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e4101e63-736b-4a05-bf77-6a97bb131c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:53.379169 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1104/2200] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:53.384485 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68561979-e387-4e95-9cd1-b9991f3486e7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:53.463969 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68561979-e387-4e95-9cd1-b9991f3486e7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:53.469831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc285935-29d5-48fb-af40-583a5c6346a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:53.470937 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc285935-29d5-48fb-af40-583a5c6346a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:53.471257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc285935-29d5-48fb-af40-583a5c6346a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:53.471556 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc285935-29d5-48fb-af40-583a5c6346a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:53.472392 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc285935-29d5-48fb-af40-583a5c6346a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:53.474664 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc285935-29d5-48fb-af40-583a5c6346a5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:53.487818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:53.514625 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1097/2201] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:53 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:53.516230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4101e63-736b-4a05-bf77-6a97bb131c6b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:53.527685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4101e63-736b-4a05-bf77-6a97bb131c6b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:53.568023 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:53.595746 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3deaa5-2c8f-4d29-a4d4-25d81efa8fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:53.596426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3deaa5-2c8f-4d29-a4d4-25d81efa8fd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:53.596706 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3deaa5-2c8f-4d29-a4d4-25d81efa8fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:53.596979 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3deaa5-2c8f-4d29-a4d4-25d81efa8fd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:53.597669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3deaa5-2c8f-4d29-a4d4-25d81efa8fd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:53.599855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3deaa5-2c8f-4d29-a4d4-25d81efa8fd7 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:53.622807 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1105/2202] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:53 2022] POST /identity/v3/users => generated 299 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:53.627103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc285935-29d5-48fb-af40-583a5c6346a5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:53.637824 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc285935-29d5-48fb-af40-583a5c6346a5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:53.642962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d945e-f0cd-4ab9-9aa9-c838d7b93c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:53.643649 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d945e-f0cd-4ab9-9aa9-c838d7b93c3a admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:53.643937 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d945e-f0cd-4ab9-9aa9-c838d7b93c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:53.644212 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d945e-f0cd-4ab9-9aa9-c838d7b93c3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:53.644996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2d945e-f0cd-4ab9-9aa9-c838d7b93c3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:53.647248 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2d945e-f0cd-4ab9-9aa9-c838d7b93c3a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:53.659264 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1098/2203] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:22:53.664525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3deaa5-2c8f-4d29-a4d4-25d81efa8fd7 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:53.675885 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3deaa5-2c8f-4d29-a4d4-25d81efa8fd7 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:53.681248 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd5b25-3c82-42bf-85cc-a2a2996e6a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:53.682171 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd5b25-3c82-42bf-85cc-a2a2996e6a2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:53.682453 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd5b25-3c82-42bf-85cc-a2a2996e6a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:53.682726 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd5b25-3c82-42bf-85cc-a2a2996e6a2a admin admin] PATH_INFO: `/v3/projects/82ef8fdeaf224ff9bd8f84c7ff9c31cc/users/4e2740c5cfba43dbae569b72d72b6847/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:53.698617 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd5b25-3c82-42bf-85cc-a2a2996e6a2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ef8fdeaf224ff9bd8f84c7ff9c31cc, user_id=4e2740c5cfba43dbae569b72d72b6847, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:53.701669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd5b25-3c82-42bf-85cc-a2a2996e6a2a admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:53.714258 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1106/2204] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:53 2022] PUT /identity/v3/projects/82ef8fdeaf224ff9bd8f84c7ff9c31cc/users/4e2740c5cfba43dbae569b72d72b6847/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:53.718242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d945e-f0cd-4ab9-9aa9-c838d7b93c3a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:53.728379 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d945e-f0cd-4ab9-9aa9-c838d7b93c3a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:53.787785 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba193e5-3f52-4d10-b7b0-e94767fd4485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:53.788375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba193e5-3f52-4d10-b7b0-e94767fd4485 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:53.788658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba193e5-3f52-4d10-b7b0-e94767fd4485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:53.788934 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba193e5-3f52-4d10-b7b0-e94767fd4485 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:53.789687 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba193e5-3f52-4d10-b7b0-e94767fd4485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:53.791320 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba193e5-3f52-4d10-b7b0-e94767fd4485 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:53.801571 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1099/2205] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:53 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:53.805858 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd5b25-3c82-42bf-85cc-a2a2996e6a2a admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:53.817014 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd5b25-3c82-42bf-85cc-a2a2996e6a2a admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:53.821571 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18c91b-8306-4d2b-beb8-46ca893fdf91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:53.822443 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18c91b-8306-4d2b-beb8-46ca893fdf91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:53.822722 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18c91b-8306-4d2b-beb8-46ca893fdf91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:53.823049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18c91b-8306-4d2b-beb8-46ca893fdf91 admin admin] PATH_INFO: `/v3/projects/82ef8fdeaf224ff9bd8f84c7ff9c31cc/users/4e2740c5cfba43dbae569b72d72b6847/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:53.825209 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd18c91b-8306-4d2b-beb8-46ca893fdf91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ef8fdeaf224ff9bd8f84c7ff9c31cc, user_id=4e2740c5cfba43dbae569b72d72b6847, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:53.828476 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd18c91b-8306-4d2b-beb8-46ca893fdf91 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:53.839426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1107/2206] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:53 2022] PUT /identity/v3/projects/82ef8fdeaf224ff9bd8f84c7ff9c31cc/users/4e2740c5cfba43dbae569b72d72b6847/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:53.851931 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488837df-d3ad-489c-a04c-c884000838f1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:53.852338 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488837df-d3ad-489c-a04c-c884000838f1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:53.852564 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488837df-d3ad-489c-a04c-c884000838f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:53.883328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-488837df-d3ad-489c-a04c-c884000838f1 None None] MFA Rules not processed for user `4e2740c5cfba43dbae569b72d72b6847`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:53.884535 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-488837df-d3ad-489c-a04c-c884000838f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:53.951694 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1100/2207] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:53.963362 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18c91b-8306-4d2b-beb8-46ca893fdf91 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:54.022270 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18c91b-8306-4d2b-beb8-46ca893fdf91 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:54.025886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5063518-b32d-48a9-957a-1098d211a62b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:54.026522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5063518-b32d-48a9-957a-1098d211a62b service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:54.026801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5063518-b32d-48a9-957a-1098d211a62b service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:54.027072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5063518-b32d-48a9-957a-1098d211a62b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:54.084951 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5063518-b32d-48a9-957a-1098d211a62b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:54.086335 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5063518-b32d-48a9-957a-1098d211a62b service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:54.095426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1108/2208] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:56.029785 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488837df-d3ad-489c-a04c-c884000838f1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:56.040627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488837df-d3ad-489c-a04c-c884000838f1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:56.045330 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11018c88-fab5-49cc-aef6-7c8ce1eedec4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:56.045982 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11018c88-fab5-49cc-aef6-7c8ce1eedec4 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:56.046267 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11018c88-fab5-49cc-aef6-7c8ce1eedec4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:56.046543 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11018c88-fab5-49cc-aef6-7c8ce1eedec4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:56.124381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11018c88-fab5-49cc-aef6-7c8ce1eedec4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:56.127178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11018c88-fab5-49cc-aef6-7c8ce1eedec4 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:56.129821 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:56.147478 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1101/2209] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:56.166043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5063518-b32d-48a9-957a-1098d211a62b service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:56.232708 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:56.253525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5063518-b32d-48a9-957a-1098d211a62b service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:56.258368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3e878-cae4-4c8a-91bf-4922a28c16a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:56.259336 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3e878-cae4-4c8a-91bf-4922a28c16a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:56.259682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3e878-cae4-4c8a-91bf-4922a28c16a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:56.260051 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3e878-cae4-4c8a-91bf-4922a28c16a1 admin admin] PATH_INFO: `/v3/users/4df6444245864bfba9093644528f77ec` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:56.261315 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3e878-cae4-4c8a-91bf-4922a28c16a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4df6444245864bfba9093644528f77ec)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:56.263745 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3e878-cae4-4c8a-91bf-4922a28c16a1 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:56.332630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-23c3e878-cae4-4c8a-91bf-4922a28c16a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4df6444245864bfba9093644528f77ec'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:56.353260 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-23c3e878-cae4-4c8a-91bf-4922a28c16a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4df6444245864bfba9093644528f77ec'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:56.354322 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-23c3e878-cae4-4c8a-91bf-4922a28c16a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4df6444245864bfba9093644528f77ec'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:56.355149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-23c3e878-cae4-4c8a-91bf-4922a28c16a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4df6444245864bfba9093644528f77ec'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:56.366281 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-23c3e878-cae4-4c8a-91bf-4922a28c16a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4df6444245864bfba9093644528f77ec'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:56.385620 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1109/2210] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:56 2022] DELETE /identity/v3/users/4df6444245864bfba9093644528f77ec => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:56.391571 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a96f24-0080-44c5-bff4-161dfc3a61de None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:56.391864 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a96f24-0080-44c5-bff4-161dfc3a61de None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:56.392147 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a96f24-0080-44c5-bff4-161dfc3a61de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:56.414879 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-85a96f24-0080-44c5-bff4-161dfc3a61de None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:56.416362 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-85a96f24-0080-44c5-bff4-161dfc3a61de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:56.498673 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1102/2211] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:56.508707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3e878-cae4-4c8a-91bf-4922a28c16a1 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:56.510137 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-23c3e878-cae4-4c8a-91bf-4922a28c16a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:56.588514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3e878-cae4-4c8a-91bf-4922a28c16a1 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:56.595146 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32bfcb-a3e9-4f7d-9c4c-555283048a9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:56.595931 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32bfcb-a3e9-4f7d-9c4c-555283048a9d service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:56.596222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32bfcb-a3e9-4f7d-9c4c-555283048a9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:56.596499 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32bfcb-a3e9-4f7d-9c4c-555283048a9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:56.609414 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32bfcb-a3e9-4f7d-9c4c-555283048a9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:56.611930 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32bfcb-a3e9-4f7d-9c4c-555283048a9d service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:56.626660 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1110/2212] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:56.832327 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a96f24-0080-44c5-bff4-161dfc3a61de None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:56.833819 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-85a96f24-0080-44c5-bff4-161dfc3a61de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:56.896585 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a96f24-0080-44c5-bff4-161dfc3a61de None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:56.900234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec57e80-37d7-48ef-980c-4461a588d52f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:56.900936 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec57e80-37d7-48ef-980c-4461a588d52f service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:56.901217 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec57e80-37d7-48ef-980c-4461a588d52f service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:56.901487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec57e80-37d7-48ef-980c-4461a588d52f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:56.904285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fec57e80-37d7-48ef-980c-4461a588d52f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:56.972815 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec57e80-37d7-48ef-980c-4461a588d52f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:56.974756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec57e80-37d7-48ef-980c-4461a588d52f service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:56.982169 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32bfcb-a3e9-4f7d-9c4c-555283048a9d service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:56.984798 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2f32bfcb-a3e9-4f7d-9c4c-555283048a9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:56.988379 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1103/2213] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:22:57.002075 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32bfcb-a3e9-4f7d-9c4c-555283048a9d service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:57.007008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45410456-eb9e-49a2-9422-587f3d870a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:57.007906 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45410456-eb9e-49a2-9422-587f3d870a07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:57.008196 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45410456-eb9e-49a2-9422-587f3d870a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:57.008482 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45410456-eb9e-49a2-9422-587f3d870a07 admin admin] PATH_INFO: `/v3/projects/1cd5dfddd24c42779f9950fc18957ac5` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:57.009765 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45410456-eb9e-49a2-9422-587f3d870a07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cd5dfddd24c42779f9950fc18957ac5)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:57.011690 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45410456-eb9e-49a2-9422-587f3d870a07 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:57.064521 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-45410456-eb9e-49a2-9422-587f3d870a07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cd5dfddd24c42779f9950fc18957ac5', 'request_id': 'req-45410456-eb9e-49a2-9422-587f3d870a07'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:57.072031 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-45410456-eb9e-49a2-9422-587f3d870a07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cd5dfddd24c42779f9950fc18957ac5', 'request_id': 'req-45410456-eb9e-49a2-9422-587f3d870a07'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:57.078754 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-45410456-eb9e-49a2-9422-587f3d870a07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cd5dfddd24c42779f9950fc18957ac5', 'request_id': 'req-45410456-eb9e-49a2-9422-587f3d870a07'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:57.079999 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:57.092751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-45410456-eb9e-49a2-9422-587f3d870a07 admin admin] The token cache is being invalidate because project 1cd5dfddd24c42779f9950fc18957ac5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:57.093065 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-45410456-eb9e-49a2-9422-587f3d870a07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:57.093727 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-45410456-eb9e-49a2-9422-587f3d870a07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:57.095347 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1111/2214] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:56 2022] DELETE /identity/v3/projects/1cd5dfddd24c42779f9950fc18957ac5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:57.099648 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec57e80-37d7-48ef-980c-4461a588d52f service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:57.110474 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec57e80-37d7-48ef-980c-4461a588d52f service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:57.155790 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:57.167949 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb437b94-c8cf-431f-9c3b-c0bf8bed76f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:57.169134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb437b94-c8cf-431f-9c3b-c0bf8bed76f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:57.169423 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb437b94-c8cf-431f-9c3b-c0bf8bed76f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:57.169728 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb437b94-c8cf-431f-9c3b-c0bf8bed76f4 admin admin] PATH_INFO: `/v3/users/a3049952da084732b189ab26281f2a02` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:57.170960 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb437b94-c8cf-431f-9c3b-c0bf8bed76f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3049952da084732b189ab26281f2a02)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:57.172953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb437b94-c8cf-431f-9c3b-c0bf8bed76f4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:57.219784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cb437b94-c8cf-431f-9c3b-c0bf8bed76f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3049952da084732b189ab26281f2a02'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:57.220434 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cb437b94-c8cf-431f-9c3b-c0bf8bed76f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3049952da084732b189ab26281f2a02'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:57.221041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cb437b94-c8cf-431f-9c3b-c0bf8bed76f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3049952da084732b189ab26281f2a02'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:57.228313 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cb437b94-c8cf-431f-9c3b-c0bf8bed76f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3049952da084732b189ab26281f2a02'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:57.240035 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cb437b94-c8cf-431f-9c3b-c0bf8bed76f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3049952da084732b189ab26281f2a02'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:57.253242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1104/2215] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:22:57 2022] DELETE /identity/v3/users/a3049952da084732b189ab26281f2a02 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:57.527517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45410456-eb9e-49a2-9422-587f3d870a07 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:57.530852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-45410456-eb9e-49a2-9422-587f3d870a07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:57.557061 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45410456-eb9e-49a2-9422-587f3d870a07 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:57.623997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff390909-3f13-40b3-a915-5187a1283c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:57.624945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff390909-3f13-40b3-a915-5187a1283c75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:57.625227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff390909-3f13-40b3-a915-5187a1283c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:57.626054 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff390909-3f13-40b3-a915-5187a1283c75 admin admin] PATH_INFO: `/v3/projects/4e33eaf5dc5148bab6abd57a63b07018` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:57.626811 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff390909-3f13-40b3-a915-5187a1283c75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e33eaf5dc5148bab6abd57a63b07018)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:57.628937 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff390909-3f13-40b3-a915-5187a1283c75 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:57.703423 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ff390909-3f13-40b3-a915-5187a1283c75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e33eaf5dc5148bab6abd57a63b07018', 'request_id': 'req-ff390909-3f13-40b3-a915-5187a1283c75'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:57.710623 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ff390909-3f13-40b3-a915-5187a1283c75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e33eaf5dc5148bab6abd57a63b07018', 'request_id': 'req-ff390909-3f13-40b3-a915-5187a1283c75'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:57.717305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ff390909-3f13-40b3-a915-5187a1283c75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e33eaf5dc5148bab6abd57a63b07018', 'request_id': 'req-ff390909-3f13-40b3-a915-5187a1283c75'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:57.727924 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ff390909-3f13-40b3-a915-5187a1283c75 admin admin] The token cache is being invalidate because project 4e33eaf5dc5148bab6abd57a63b07018 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:22:57.728220 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ff390909-3f13-40b3-a915-5187a1283c75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:57.728852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ff390909-3f13-40b3-a915-5187a1283c75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:22:57.730386 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1112/2216] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:22:57 2022] DELETE /identity/v3/projects/4e33eaf5dc5148bab6abd57a63b07018 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:57.736006 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737c185-577f-470a-ab85-f2ba51ff5114 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:57.736301 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737c185-577f-470a-ab85-f2ba51ff5114 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:57.736580 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737c185-577f-470a-ab85-f2ba51ff5114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:57.753908 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e737c185-577f-470a-ab85-f2ba51ff5114 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:57.755106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e737c185-577f-470a-ab85-f2ba51ff5114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:57.817927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1105/2217] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:57.827199 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6d902-3fbc-4e0a-999c-6c995bc3b61b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:57.827655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6d902-3fbc-4e0a-999c-6c995bc3b61b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:57.827999 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6d902-3fbc-4e0a-999c-6c995bc3b61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:57.847552 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-24c6d902-3fbc-4e0a-999c-6c995bc3b61b None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:57.848769 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-24c6d902-3fbc-4e0a-999c-6c995bc3b61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:57.857854 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1113/2218] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:57.862082 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737c185-577f-470a-ab85-f2ba51ff5114 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:57.871750 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737c185-577f-470a-ab85-f2ba51ff5114 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:57.876291 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d26ce3a-fe5d-4195-8ac0-e74582797ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:57.876951 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26ce3a-fe5d-4195-8ac0-e74582797ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:57.877229 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26ce3a-fe5d-4195-8ac0-e74582797ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:57.877497 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26ce3a-fe5d-4195-8ac0-e74582797ae5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:57.878223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d26ce3a-fe5d-4195-8ac0-e74582797ae5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:22:57.878586 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d26ce3a-fe5d-4195-8ac0-e74582797ae5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:57.880200 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d26ce3a-fe5d-4195-8ac0-e74582797ae5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:57.892015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1106/2219] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:22:57.896190 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6d902-3fbc-4e0a-999c-6c995bc3b61b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:57.905962 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6d902-3fbc-4e0a-999c-6c995bc3b61b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:57.910786 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179f161-513e-40df-99c8-89ea7673c498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:57.911396 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179f161-513e-40df-99c8-89ea7673c498 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:57.911681 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179f161-513e-40df-99c8-89ea7673c498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:57.911957 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179f161-513e-40df-99c8-89ea7673c498 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:57.912676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179f161-513e-40df-99c8-89ea7673c498 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:57.914576 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179f161-513e-40df-99c8-89ea7673c498 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:57.941999 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1114/2220] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:22:57 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:57.946182 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d26ce3a-fe5d-4195-8ac0-e74582797ae5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:57.956601 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d26ce3a-fe5d-4195-8ac0-e74582797ae5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:58.009756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:58.025433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300c8a8-aed5-4967-98f3-03f27906bf06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:58.026135 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300c8a8-aed5-4967-98f3-03f27906bf06 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:58.026398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300c8a8-aed5-4967-98f3-03f27906bf06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:58.026676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300c8a8-aed5-4967-98f3-03f27906bf06 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:58.027408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300c8a8-aed5-4967-98f3-03f27906bf06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:58.033907 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300c8a8-aed5-4967-98f3-03f27906bf06 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:58.055419 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1107/2221] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:22:57 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:58.059786 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179f161-513e-40df-99c8-89ea7673c498 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:58.070237 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179f161-513e-40df-99c8-89ea7673c498 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:58.075460 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9e170-d9b3-4fda-b0e1-408f5385558e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:58.076120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9e170-d9b3-4fda-b0e1-408f5385558e admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:58.076401 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9e170-d9b3-4fda-b0e1-408f5385558e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:58.076675 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9e170-d9b3-4fda-b0e1-408f5385558e admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:58.077402 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9e170-d9b3-4fda-b0e1-408f5385558e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:58.078984 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9e170-d9b3-4fda-b0e1-408f5385558e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:58.089410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1115/2222] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:58.094051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300c8a8-aed5-4967-98f3-03f27906bf06 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:58.104489 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300c8a8-aed5-4967-98f3-03f27906bf06 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:58.109378 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b327d8de-a866-4a3c-88fe-e16f81bd2c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:58.110070 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327d8de-a866-4a3c-88fe-e16f81bd2c38 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:58.110381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327d8de-a866-4a3c-88fe-e16f81bd2c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:58.110658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327d8de-a866-4a3c-88fe-e16f81bd2c38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:58.111420 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b327d8de-a866-4a3c-88fe-e16f81bd2c38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:58.113018 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b327d8de-a866-4a3c-88fe-e16f81bd2c38 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:58.124371 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1108/2223] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:22:58.128624 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9e170-d9b3-4fda-b0e1-408f5385558e admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:58.139985 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9e170-d9b3-4fda-b0e1-408f5385558e admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:58.144084 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19c109-5fb4-4494-adf6-a6682f4f1815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:58.144962 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19c109-5fb4-4494-adf6-a6682f4f1815 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:58.145241 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19c109-5fb4-4494-adf6-a6682f4f1815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:58.145512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19c109-5fb4-4494-adf6-a6682f4f1815 admin admin] PATH_INFO: `/v3/projects/c5513a83ea324731b26e7223a468f382/users/85445672557848d5a377682b9cdeee61/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:58.160517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19c109-5fb4-4494-adf6-a6682f4f1815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5513a83ea324731b26e7223a468f382, user_id=85445672557848d5a377682b9cdeee61, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:58.163194 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19c109-5fb4-4494-adf6-a6682f4f1815 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:58.176806 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1116/2224] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:22:58 2022] PUT /identity/v3/projects/c5513a83ea324731b26e7223a468f382/users/85445672557848d5a377682b9cdeee61/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:22:58.188114 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df7e1f-9f74-4928-b252-8de29caf35e2 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:58.188406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df7e1f-9f74-4928-b252-8de29caf35e2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:58.188688 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df7e1f-9f74-4928-b252-8de29caf35e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:58.214963 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b7df7e1f-9f74-4928-b252-8de29caf35e2 None None] MFA Rules not processed for user `85445672557848d5a377682b9cdeee61`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:58.216784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b7df7e1f-9f74-4928-b252-8de29caf35e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:58.281568 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1109/2225] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:22:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:22:58.292319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19c109-5fb4-4494-adf6-a6682f4f1815 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:58.294682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4e19c109-5fb4-4494-adf6-a6682f4f1815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:58.312974 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:58.354569 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19c109-5fb4-4494-adf6-a6682f4f1815 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:58.358301 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b608e6f-3004-43df-a326-f82c046b71b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:58.358940 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b608e6f-3004-43df-a326-f82c046b71b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:58.359236 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b608e6f-3004-43df-a326-f82c046b71b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:58.359527 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b608e6f-3004-43df-a326-f82c046b71b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:58.367884 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b608e6f-3004-43df-a326-f82c046b71b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:58.369309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b608e6f-3004-43df-a326-f82c046b71b9 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:58.378503 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1117/2226] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:22:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:23:06.312936 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df7e1f-9f74-4928-b252-8de29caf35e2 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:06.314771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b7df7e1f-9f74-4928-b252-8de29caf35e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:06.393101 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b608e6f-3004-43df-a326-f82c046b71b9 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:06.394645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4b608e6f-3004-43df-a326-f82c046b71b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:06.425800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df7e1f-9f74-4928-b252-8de29caf35e2 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:06.430488 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd52340-3620-4331-ac66-91f7a02481aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:06.431613 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd52340-3620-4331-ac66-91f7a02481aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:06.432051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd52340-3620-4331-ac66-91f7a02481aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:06.432327 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd52340-3620-4331-ac66-91f7a02481aa admin admin] PATH_INFO: `/v3/users/fe9f703e7c704f2884f010be93db638c` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:06.433541 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd52340-3620-4331-ac66-91f7a02481aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe9f703e7c704f2884f010be93db638c)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:06.436068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd52340-3620-4331-ac66-91f7a02481aa admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:06.479151 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b608e6f-3004-43df-a326-f82c046b71b9 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:06.486198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d66c8-8c37-487c-9593-d3fb6a64c88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:06.487184 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d66c8-8c37-487c-9593-d3fb6a64c88f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:06.487477 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d66c8-8c37-487c-9593-d3fb6a64c88f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:06.487810 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d66c8-8c37-487c-9593-d3fb6a64c88f admin admin] PATH_INFO: `/v3/users/140093d4082342e2b4178781f09e643d` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:06.489486 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d66c8-8c37-487c-9593-d3fb6a64c88f admin admin] RBAC: Authorizing `identity:delete_user(user_id=140093d4082342e2b4178781f09e643d)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:06.491491 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d66c8-8c37-487c-9593-d3fb6a64c88f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:06.492134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3dd52340-3620-4331-ac66-91f7a02481aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe9f703e7c704f2884f010be93db638c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:06.493085 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3dd52340-3620-4331-ac66-91f7a02481aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe9f703e7c704f2884f010be93db638c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:06.493989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3dd52340-3620-4331-ac66-91f7a02481aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe9f703e7c704f2884f010be93db638c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:06.502741 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3dd52340-3620-4331-ac66-91f7a02481aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe9f703e7c704f2884f010be93db638c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:06.510520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:06.529488 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3dd52340-3620-4331-ac66-91f7a02481aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe9f703e7c704f2884f010be93db638c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:06.559210 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1110/2227] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:23:06 2022] DELETE /identity/v3/users/fe9f703e7c704f2884f010be93db638c => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:06.563725 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2db2bb-c138-4f14-a9cb-b039d1baeebf None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:06.564092 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2db2bb-c138-4f14-a9cb-b039d1baeebf None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:06.564494 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2db2bb-c138-4f14-a9cb-b039d1baeebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:06.565316 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1a9d66c8-8c37-487c-9593-d3fb6a64c88f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '140093d4082342e2b4178781f09e643d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:06.586378 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1a9d66c8-8c37-487c-9593-d3fb6a64c88f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '140093d4082342e2b4178781f09e643d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:06.588223 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1a9d66c8-8c37-487c-9593-d3fb6a64c88f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '140093d4082342e2b4178781f09e643d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:06.589480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1a9d66c8-8c37-487c-9593-d3fb6a64c88f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '140093d4082342e2b4178781f09e643d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:06.591171 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5f2db2bb-c138-4f14-a9cb-b039d1baeebf None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:06.593109 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5f2db2bb-c138-4f14-a9cb-b039d1baeebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:06.604610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1a9d66c8-8c37-487c-9593-d3fb6a64c88f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '140093d4082342e2b4178781f09e643d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:06.634565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1118/2228] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:23:06 2022] DELETE /identity/v3/users/140093d4082342e2b4178781f09e643d => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:23:06.640687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5aa6b3-249c-4a98-94f8-d1ab640b7539 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:06.640974 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5aa6b3-249c-4a98-94f8-d1ab640b7539 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:06.641248 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5aa6b3-249c-4a98-94f8-d1ab640b7539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:06.668660 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-3a5aa6b3-249c-4a98-94f8-d1ab640b7539 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:06.670301 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3a5aa6b3-249c-4a98-94f8-d1ab640b7539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:06.680682 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1111/2229] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:23:06.692002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2db2bb-c138-4f14-a9cb-b039d1baeebf None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:06.694917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5f2db2bb-c138-4f14-a9cb-b039d1baeebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:06.714857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:06.772172 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1119/2230] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:06.779149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5aa6b3-249c-4a98-94f8-d1ab640b7539 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:06.789462 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2db2bb-c138-4f14-a9cb-b039d1baeebf None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:06.882692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5aa6b3-249c-4a98-94f8-d1ab640b7539 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:06.888092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0dd543-b450-4c3a-b653-d873610d6b8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:06.891672 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dd543-b450-4c3a-b653-d873610d6b8f service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:06.891981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dd543-b450-4c3a-b653-d873610d6b8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:06.892241 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dd543-b450-4c3a-b653-d873610d6b8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:06.895487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ee397-4254-4915-8ade-1f2da5136a6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:06.896188 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ee397-4254-4915-8ade-1f2da5136a6d service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:06.897802 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ee397-4254-4915-8ade-1f2da5136a6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:06.898258 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ee397-4254-4915-8ade-1f2da5136a6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:06.923630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0dd543-b450-4c3a-b653-d873610d6b8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:06.925257 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0dd543-b450-4c3a-b653-d873610d6b8f service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:06.929993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314ee397-4254-4915-8ade-1f2da5136a6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:06.932200 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314ee397-4254-4915-8ade-1f2da5136a6d service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:06.953847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1120/2231] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:23:06.959978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1112/2232] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:23:07.388670 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0dd543-b450-4c3a-b653-d873610d6b8f service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:07.390300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6a0dd543-b450-4c3a-b653-d873610d6b8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:07.418856 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0dd543-b450-4c3a-b653-d873610d6b8f service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:07.424020 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef19a4-782a-4684-8245-051f9b97238f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:07.424981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef19a4-782a-4684-8245-051f9b97238f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:07.425332 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef19a4-782a-4684-8245-051f9b97238f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:07.425672 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef19a4-782a-4684-8245-051f9b97238f admin admin] PATH_INFO: `/v3/projects/f9ed7398854449739ad5be06f7a35441` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:07.426937 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef19a4-782a-4684-8245-051f9b97238f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9ed7398854449739ad5be06f7a35441)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:07.428999 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef19a4-782a-4684-8245-051f9b97238f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:07.463944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ee397-4254-4915-8ade-1f2da5136a6d service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:07.466272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-314ee397-4254-4915-8ade-1f2da5136a6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:07.486256 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ee397-4254-4915-8ade-1f2da5136a6d service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:07.495429 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-bfef19a4-782a-4684-8245-051f9b97238f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9ed7398854449739ad5be06f7a35441', 'request_id': 'req-bfef19a4-782a-4684-8245-051f9b97238f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:07.505271 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-bfef19a4-782a-4684-8245-051f9b97238f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9ed7398854449739ad5be06f7a35441', 'request_id': 'req-bfef19a4-782a-4684-8245-051f9b97238f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:07.512780 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-bfef19a4-782a-4684-8245-051f9b97238f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9ed7398854449739ad5be06f7a35441', 'request_id': 'req-bfef19a4-782a-4684-8245-051f9b97238f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:07.524139 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-bfef19a4-782a-4684-8245-051f9b97238f admin admin] The token cache is being invalidate because project f9ed7398854449739ad5be06f7a35441 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:23:07.524645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-bfef19a4-782a-4684-8245-051f9b97238f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:07.525326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-bfef19a4-782a-4684-8245-051f9b97238f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:07.526895 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1121/2233] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:23:07 2022] DELETE /identity/v3/projects/f9ed7398854449739ad5be06f7a35441 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:07.531195 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef19a4-782a-4684-8245-051f9b97238f admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:07.532581 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-bfef19a4-782a-4684-8245-051f9b97238f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:07.534737 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:07.555364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ed0f4-03b0-4afd-8bde-f29ad958fb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:07.556304 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ed0f4-03b0-4afd-8bde-f29ad958fb82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:07.556611 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ed0f4-03b0-4afd-8bde-f29ad958fb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:07.556861 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ed0f4-03b0-4afd-8bde-f29ad958fb82 admin admin] PATH_INFO: `/v3/projects/8cfdb8a057924628a5a46ad4d5124470` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:07.558231 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ed0f4-03b0-4afd-8bde-f29ad958fb82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cfdb8a057924628a5a46ad4d5124470)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:07.561243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ed0f4-03b0-4afd-8bde-f29ad958fb82 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:07.608999 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef19a4-782a-4684-8245-051f9b97238f admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:07.617688 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:07.618731 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-032ed0f4-03b0-4afd-8bde-f29ad958fb82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cfdb8a057924628a5a46ad4d5124470', 'request_id': 'req-032ed0f4-03b0-4afd-8bde-f29ad958fb82'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:07.628756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-032ed0f4-03b0-4afd-8bde-f29ad958fb82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cfdb8a057924628a5a46ad4d5124470', 'request_id': 'req-032ed0f4-03b0-4afd-8bde-f29ad958fb82'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:07.635986 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-032ed0f4-03b0-4afd-8bde-f29ad958fb82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cfdb8a057924628a5a46ad4d5124470', 'request_id': 'req-032ed0f4-03b0-4afd-8bde-f29ad958fb82'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:07.649117 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-032ed0f4-03b0-4afd-8bde-f29ad958fb82 admin admin] The token cache is being invalidate because project 8cfdb8a057924628a5a46ad4d5124470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:23:07.649398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-032ed0f4-03b0-4afd-8bde-f29ad958fb82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:07.650052 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-032ed0f4-03b0-4afd-8bde-f29ad958fb82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:07.652562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1113/2234] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:23:07 2022] DELETE /identity/v3/projects/8cfdb8a057924628a5a46ad4d5124470 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:23:07.657911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4fce2-546e-4757-b92c-baacb29562a9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:07.657911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4fce2-546e-4757-b92c-baacb29562a9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:07.658209 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4fce2-546e-4757-b92c-baacb29562a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:07.674402 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccfc190-ada0-4b6c-bfd4-57074500ee07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:07.675557 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfc190-ada0-4b6c-bfd4-57074500ee07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:07.675966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfc190-ada0-4b6c-bfd4-57074500ee07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:07.676288 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c4f4fce2-546e-4757-b92c-baacb29562a9 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:07.676448 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfc190-ada0-4b6c-bfd4-57074500ee07 admin admin] PATH_INFO: `/v3/users/d91830e475fb4c5fae266ca583d935f9` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:07.677598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c4f4fce2-546e-4757-b92c-baacb29562a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:07.678059 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccfc190-ada0-4b6c-bfd4-57074500ee07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d91830e475fb4c5fae266ca583d935f9)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:07.680080 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccfc190-ada0-4b6c-bfd4-57074500ee07 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:07.687690 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1114/2235] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:23:07.696370 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27d905-6ce0-4c76-83bf-2ae638cec97d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:07.696656 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27d905-6ce0-4c76-83bf-2ae638cec97d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:07.696936 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27d905-6ce0-4c76-83bf-2ae638cec97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:07.717345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-af27d905-6ce0-4c76-83bf-2ae638cec97d None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:07.718630 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-af27d905-6ce0-4c76-83bf-2ae638cec97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:07.727469 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1115/2236] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:07.731349 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-2ccfc190-ada0-4b6c-bfd4-57074500ee07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd91830e475fb4c5fae266ca583d935f9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:07.732112 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27d905-6ce0-4c76-83bf-2ae638cec97d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:07.741408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27d905-6ce0-4c76-83bf-2ae638cec97d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:07.744727 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-2ccfc190-ada0-4b6c-bfd4-57074500ee07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd91830e475fb4c5fae266ca583d935f9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:07.745432 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-2ccfc190-ada0-4b6c-bfd4-57074500ee07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd91830e475fb4c5fae266ca583d935f9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:07.746610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-2ccfc190-ada0-4b6c-bfd4-57074500ee07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd91830e475fb4c5fae266ca583d935f9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:07.746861 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d09df4-dadc-4c7a-9f09-ddb0d96915ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:07.747611 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d09df4-dadc-4c7a-9f09-ddb0d96915ba admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:07.747925 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d09df4-dadc-4c7a-9f09-ddb0d96915ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:07.748218 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d09df4-dadc-4c7a-9f09-ddb0d96915ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:07.749214 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d09df4-dadc-4c7a-9f09-ddb0d96915ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:07.749214 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d09df4-dadc-4c7a-9f09-ddb0d96915ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:07.750865 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d09df4-dadc-4c7a-9f09-ddb0d96915ba admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:07.754921 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-2ccfc190-ada0-4b6c-bfd4-57074500ee07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd91830e475fb4c5fae266ca583d935f9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:07.767388 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1116/2237] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:07.769270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d09df4-dadc-4c7a-9f09-ddb0d96915ba admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:07.770089 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1122/2238] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:23:07 2022] DELETE /identity/v3/users/d91830e475fb4c5fae266ca583d935f9 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:07.771135 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-50d09df4-dadc-4c7a-9f09-ddb0d96915ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:07.785638 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d09df4-dadc-4c7a-9f09-ddb0d96915ba admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:07.852700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b0ed7-4f83-4bf7-a363-eecd838183e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:07.853478 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b0ed7-4f83-4bf7-a363-eecd838183e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:07.853794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b0ed7-4f83-4bf7-a363-eecd838183e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:07.854074 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b0ed7-4f83-4bf7-a363-eecd838183e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:07.854804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b0ed7-4f83-4bf7-a363-eecd838183e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:07.857700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b0ed7-4f83-4bf7-a363-eecd838183e9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:07.886027 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1117/2239] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:23:07 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:23:07.890503 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccfc190-ada0-4b6c-bfd4-57074500ee07 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:07.901156 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccfc190-ada0-4b6c-bfd4-57074500ee07 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:07.992579 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ee82e-e805-4489-b0f3-016b64642155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:07.994846 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ee82e-e805-4489-b0f3-016b64642155 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:07.995277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ee82e-e805-4489-b0f3-016b64642155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:07.995733 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ee82e-e805-4489-b0f3-016b64642155 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:07.997088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ee82e-e805-4489-b0f3-016b64642155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:07.999109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ee82e-e805-4489-b0f3-016b64642155 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:08.023821 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1123/2240] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:23:07 2022] POST /identity/v3/users => generated 288 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:08.029208 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b0ed7-4f83-4bf7-a363-eecd838183e9 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:08.041284 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b0ed7-4f83-4bf7-a363-eecd838183e9 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:08.046999 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417d987-0d1f-41c9-b48e-5b523e7f7de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:08.047831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417d987-0d1f-41c9-b48e-5b523e7f7de1 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:08.048088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417d987-0d1f-41c9-b48e-5b523e7f7de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:08.048369 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417d987-0d1f-41c9-b48e-5b523e7f7de1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:08.049218 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417d987-0d1f-41c9-b48e-5b523e7f7de1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:08.051152 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417d987-0d1f-41c9-b48e-5b523e7f7de1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:08.063473 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1118/2241] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:23:08.068315 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ee82e-e805-4489-b0f3-016b64642155 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:08.079744 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ee82e-e805-4489-b0f3-016b64642155 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:08.088159 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcc99d-b280-45d9-bfd2-557b8619a68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:08.088976 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcc99d-b280-45d9-bfd2-557b8619a68d admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:08.089242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417d987-0d1f-41c9-b48e-5b523e7f7de1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:08.089447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcc99d-b280-45d9-bfd2-557b8619a68d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:08.090452 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcc99d-b280-45d9-bfd2-557b8619a68d admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:08.091986 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f417d987-0d1f-41c9-b48e-5b523e7f7de1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:08.093343 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bcc99d-b280-45d9-bfd2-557b8619a68d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:08.095055 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bcc99d-b280-45d9-bfd2-557b8619a68d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:08.120822 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417d987-0d1f-41c9-b48e-5b523e7f7de1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:08.126946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1124/2242] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:23:08.128270 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcc99d-b280-45d9-bfd2-557b8619a68d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:08.131705 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-23bcc99d-b280-45d9-bfd2-557b8619a68d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:08.133587 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5c7bc-b496-4b4d-b910-6526f70faab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:08.137929 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5c7bc-b496-4b4d-b910-6526f70faab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:08.138481 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5c7bc-b496-4b4d-b910-6526f70faab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:08.138833 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5c7bc-b496-4b4d-b910-6526f70faab9 admin admin] PATH_INFO: `/v3/projects/8e6befea55844949a070c7899172df7c` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:08.140511 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b5c7bc-b496-4b4d-b910-6526f70faab9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e6befea55844949a070c7899172df7c)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:08.142711 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b5c7bc-b496-4b4d-b910-6526f70faab9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:08.155756 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcc99d-b280-45d9-bfd2-557b8619a68d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:08.172762 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8de0f-e71a-4cc1-a14f-1d5db77bb948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:08.174550 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8de0f-e71a-4cc1-a14f-1d5db77bb948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:08.174944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8de0f-e71a-4cc1-a14f-1d5db77bb948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:08.175507 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8de0f-e71a-4cc1-a14f-1d5db77bb948 admin admin] PATH_INFO: `/v3/users/85445672557848d5a377682b9cdeee61` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:08.177181 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff8de0f-e71a-4cc1-a14f-1d5db77bb948 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85445672557848d5a377682b9cdeee61)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:08.179438 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff8de0f-e71a-4cc1-a14f-1d5db77bb948 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:08.182849 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:08.194266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:08.251001 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-aff8de0f-e71a-4cc1-a14f-1d5db77bb948 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85445672557848d5a377682b9cdeee61'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:08.255754 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f2b5c7bc-b496-4b4d-b910-6526f70faab9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e6befea55844949a070c7899172df7c', 'request_id': 'req-f2b5c7bc-b496-4b4d-b910-6526f70faab9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:08.268472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-aff8de0f-e71a-4cc1-a14f-1d5db77bb948 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85445672557848d5a377682b9cdeee61'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:08.268754 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-aff8de0f-e71a-4cc1-a14f-1d5db77bb948 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85445672557848d5a377682b9cdeee61'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:08.269732 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-aff8de0f-e71a-4cc1-a14f-1d5db77bb948 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85445672557848d5a377682b9cdeee61'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:08.270113 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f2b5c7bc-b496-4b4d-b910-6526f70faab9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e6befea55844949a070c7899172df7c', 'request_id': 'req-f2b5c7bc-b496-4b4d-b910-6526f70faab9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:08.276206 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f2b5c7bc-b496-4b4d-b910-6526f70faab9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e6befea55844949a070c7899172df7c', 'request_id': 'req-f2b5c7bc-b496-4b4d-b910-6526f70faab9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:08.284751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-aff8de0f-e71a-4cc1-a14f-1d5db77bb948 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85445672557848d5a377682b9cdeee61'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:08.295800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f2b5c7bc-b496-4b4d-b910-6526f70faab9 admin admin] The token cache is being invalidate because project 8e6befea55844949a070c7899172df7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:23:08.296234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f2b5c7bc-b496-4b4d-b910-6526f70faab9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:08.297102 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f2b5c7bc-b496-4b4d-b910-6526f70faab9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:08.302297 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1119/2243] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:23:08 2022] DELETE /identity/v3/projects/8e6befea55844949a070c7899172df7c => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:23:08.302297 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5c7bc-b496-4b4d-b910-6526f70faab9 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:08.304090 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f2b5c7bc-b496-4b4d-b910-6526f70faab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:08.309537 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1125/2244] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:23:08 2022] DELETE /identity/v3/users/85445672557848d5a377682b9cdeee61 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:08.310831 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bcae1-61de-4214-8822-46c95cd3eb32 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:08.311123 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bcae1-61de-4214-8822-46c95cd3eb32 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:08.311429 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bcae1-61de-4214-8822-46c95cd3eb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:08.340248 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-383bcae1-61de-4214-8822-46c95cd3eb32 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:08.341591 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-383bcae1-61de-4214-8822-46c95cd3eb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:08.398704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5c7bc-b496-4b4d-b910-6526f70faab9 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:08.473266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1126/2245] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:08.474929 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97673378-b10c-418c-a2fc-dffebfc8cd92 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:08.475254 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97673378-b10c-418c-a2fc-dffebfc8cd92 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:08.475526 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97673378-b10c-418c-a2fc-dffebfc8cd92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:08.506645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-97673378-b10c-418c-a2fc-dffebfc8cd92 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:08.507894 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-97673378-b10c-418c-a2fc-dffebfc8cd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:08.508613 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208d80a-bdc3-4393-b83d-ab0e7c217e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:08.509651 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208d80a-bdc3-4393-b83d-ab0e7c217e28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:08.509938 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208d80a-bdc3-4393-b83d-ab0e7c217e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:08.510222 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208d80a-bdc3-4393-b83d-ab0e7c217e28 admin admin] PATH_INFO: `/v3/projects/f171333b618d4e1d80411cfac5544df9/users/7e838dd1f8464ff490a791441807cd8b/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:08.522173 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1127/2246] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:08.523693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cccd91-823a-48fc-97a1-f2bfa9729dcf None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:08.524019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cccd91-823a-48fc-97a1-f2bfa9729dcf None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:08.524414 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cccd91-823a-48fc-97a1-f2bfa9729dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:08.537667 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1208d80a-bdc3-4393-b83d-ab0e7c217e28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f171333b618d4e1d80411cfac5544df9, user_id=7e838dd1f8464ff490a791441807cd8b, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:08.541584 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1208d80a-bdc3-4393-b83d-ab0e7c217e28 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:08.556901 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-d0cccd91-823a-48fc-97a1-f2bfa9729dcf None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:08.558744 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d0cccd91-823a-48fc-97a1-f2bfa9729dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:08.560909 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1120/2247] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:23:08 2022] PUT /identity/v3/projects/f171333b618d4e1d80411cfac5544df9/users/7e838dd1f8464ff490a791441807cd8b/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:08.562049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208d80a-bdc3-4393-b83d-ab0e7c217e28 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:08.575819 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1208d80a-bdc3-4393-b83d-ab0e7c217e28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:08.622577 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:08.654898 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208d80a-bdc3-4393-b83d-ab0e7c217e28 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:08.655165 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1128/2248] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:08.658801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb4a2a-e1c4-4655-8e4f-6638aca6d37c None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:08.658801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb4a2a-e1c4-4655-8e4f-6638aca6d37c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:08.658801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb4a2a-e1c4-4655-8e4f-6638aca6d37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:08.662577 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b6931-a56d-4b60-81db-9e8cbeebc92a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:08.663275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b6931-a56d-4b60-81db-9e8cbeebc92a service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:08.663582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b6931-a56d-4b60-81db-9e8cbeebc92a service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:08.664425 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b6931-a56d-4b60-81db-9e8cbeebc92a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:08.674748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b6931-a56d-4b60-81db-9e8cbeebc92a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:08.676227 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b6931-a56d-4b60-81db-9e8cbeebc92a service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:08.686176 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1121/2249] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:23:08.686879 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-42bb4a2a-e1c4-4655-8e4f-6638aca6d37c None None] MFA Rules not processed for user `7e838dd1f8464ff490a791441807cd8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:08.688886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-42bb4a2a-e1c4-4655-8e4f-6638aca6d37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:08.692337 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b6931-a56d-4b60-81db-9e8cbeebc92a service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:08.693433 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:08.705281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b6931-a56d-4b60-81db-9e8cbeebc92a service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:08.711986 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917eafdd-0483-4223-a260-b7d27b8555c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:08.712658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917eafdd-0483-4223-a260-b7d27b8555c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:08.712915 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917eafdd-0483-4223-a260-b7d27b8555c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:08.713189 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917eafdd-0483-4223-a260-b7d27b8555c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:08.714019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917eafdd-0483-4223-a260-b7d27b8555c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:08.715863 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917eafdd-0483-4223-a260-b7d27b8555c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:08.717284 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917eafdd-0483-4223-a260-b7d27b8555c3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:08.735279 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1122/2250] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:08.737471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917eafdd-0483-4223-a260-b7d27b8555c3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:08.749042 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917eafdd-0483-4223-a260-b7d27b8555c3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:08.755401 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef6bb5-6fc3-408e-8e48-7ddf06a3d822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:08.755969 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef6bb5-6fc3-408e-8e48-7ddf06a3d822 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:08.756531 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef6bb5-6fc3-408e-8e48-7ddf06a3d822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:08.756717 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef6bb5-6fc3-408e-8e48-7ddf06a3d822 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:08.757348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ef6bb5-6fc3-408e-8e48-7ddf06a3d822 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:08.759284 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ef6bb5-6fc3-408e-8e48-7ddf06a3d822 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:08.792005 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1129/2251] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:08.796185 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb4a2a-e1c4-4655-8e4f-6638aca6d37c None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:08.802369 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1123/2252] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:23:08 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:23:08.805929 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef6bb5-6fc3-408e-8e48-7ddf06a3d822 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:08.809861 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb4a2a-e1c4-4655-8e4f-6638aca6d37c None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:08.886255 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef6bb5-6fc3-408e-8e48-7ddf06a3d822 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:08.893094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834f762-f465-4d39-a4f0-2d593a813558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:08.893923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834f762-f465-4d39-a4f0-2d593a813558 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:08.894448 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834f762-f465-4d39-a4f0-2d593a813558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:08.895091 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834f762-f465-4d39-a4f0-2d593a813558 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:08.895679 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834f762-f465-4d39-a4f0-2d593a813558 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:08.897878 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834f762-f465-4d39-a4f0-2d593a813558 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:08.913592 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8675526-39aa-4ce3-a11f-ea9d36c65221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:08.914363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8675526-39aa-4ce3-a11f-ea9d36c65221 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:08.914684 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8675526-39aa-4ce3-a11f-ea9d36c65221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:08.915026 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8675526-39aa-4ce3-a11f-ea9d36c65221 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:08.915765 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8675526-39aa-4ce3-a11f-ea9d36c65221 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:08.918586 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8675526-39aa-4ce3-a11f-ea9d36c65221 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:08.927684 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1124/2253] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:23:08 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:23:08.931577 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834f762-f465-4d39-a4f0-2d593a813558 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:08.943276 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834f762-f465-4d39-a4f0-2d593a813558 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:08.950557 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1130/2254] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:23:08 2022] POST /identity/v3/projects => generated 375 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:08.950721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773c66a-c495-4c34-9cde-2ba256768852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:08.950721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773c66a-c495-4c34-9cde-2ba256768852 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:08.950721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773c66a-c495-4c34-9cde-2ba256768852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:08.950721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773c66a-c495-4c34-9cde-2ba256768852 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:08.951331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6773c66a-c495-4c34-9cde-2ba256768852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:08.951688 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8675526-39aa-4ce3-a11f-ea9d36c65221 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:08.953251 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6773c66a-c495-4c34-9cde-2ba256768852 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:08.965627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1125/2255] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:23:08.969588 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773c66a-c495-4c34-9cde-2ba256768852 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:08.981486 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773c66a-c495-4c34-9cde-2ba256768852 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:09.036832 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8675526-39aa-4ce3-a11f-ea9d36c65221 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:09.043287 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aff6e0-3538-41ca-81e4-7baa7e550d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:09.044037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aff6e0-3538-41ca-81e4-7baa7e550d22 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:09.044371 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aff6e0-3538-41ca-81e4-7baa7e550d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:09.044596 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aff6e0-3538-41ca-81e4-7baa7e550d22 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:09.045315 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aff6e0-3538-41ca-81e4-7baa7e550d22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:09.047248 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aff6e0-3538-41ca-81e4-7baa7e550d22 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:09.051521 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b11ed0-9314-4d92-ac67-5b29f9082ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:09.052184 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b11ed0-9314-4d92-ac67-5b29f9082ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:09.052464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b11ed0-9314-4d92-ac67-5b29f9082ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:09.052742 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b11ed0-9314-4d92-ac67-5b29f9082ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:09.053576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b11ed0-9314-4d92-ac67-5b29f9082ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:09.055339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b11ed0-9314-4d92-ac67-5b29f9082ab0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:09.066985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1126/2256] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:23:09.069821 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1131/2257] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:23:08 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:09.070847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aff6e0-3538-41ca-81e4-7baa7e550d22 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:09.071448 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b11ed0-9314-4d92-ac67-5b29f9082ab0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:09.073602 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-01aff6e0-3538-41ca-81e4-7baa7e550d22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:09.081984 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b11ed0-9314-4d92-ac67-5b29f9082ab0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:09.086472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aff6e0-3538-41ca-81e4-7baa7e550d22 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:09.087266 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cc1a2-60e6-436f-acb5-368879d1becb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:09.088159 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cc1a2-60e6-436f-acb5-368879d1becb admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:09.088448 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cc1a2-60e6-436f-acb5-368879d1becb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:09.088734 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cc1a2-60e6-436f-acb5-368879d1becb admin admin] PATH_INFO: `/v3/projects/ae91adf4b6f341a194bca5f0cad50437/users/e8e8763ea0a04caea7d81b816c085817/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:09.091728 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4722ac-5cc6-42f3-8214-5869cffaa041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:09.092590 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4722ac-5cc6-42f3-8214-5869cffaa041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:09.092871 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4722ac-5cc6-42f3-8214-5869cffaa041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:09.093190 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4722ac-5cc6-42f3-8214-5869cffaa041 admin admin] PATH_INFO: `/v3/projects/c5513a83ea324731b26e7223a468f382` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:09.094389 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4722ac-5cc6-42f3-8214-5869cffaa041 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5513a83ea324731b26e7223a468f382)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:09.096313 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4722ac-5cc6-42f3-8214-5869cffaa041 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:09.101060 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:09.104215 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06cc1a2-60e6-436f-acb5-368879d1becb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae91adf4b6f341a194bca5f0cad50437, user_id=e8e8763ea0a04caea7d81b816c085817, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:09.107496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06cc1a2-60e6-436f-acb5-368879d1becb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:09.115138 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:09.129633 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1127/2258] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:23:09 2022] PUT /identity/v3/projects/ae91adf4b6f341a194bca5f0cad50437/users/e8e8763ea0a04caea7d81b816c085817/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:09.129633 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cc1a2-60e6-436f-acb5-368879d1becb admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:09.160035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-aa4722ac-5cc6-42f3-8214-5869cffaa041 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5513a83ea324731b26e7223a468f382', 'request_id': 'req-aa4722ac-5cc6-42f3-8214-5869cffaa041'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:09.171338 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-aa4722ac-5cc6-42f3-8214-5869cffaa041 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5513a83ea324731b26e7223a468f382', 'request_id': 'req-aa4722ac-5cc6-42f3-8214-5869cffaa041'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:09.180105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-aa4722ac-5cc6-42f3-8214-5869cffaa041 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5513a83ea324731b26e7223a468f382', 'request_id': 'req-aa4722ac-5cc6-42f3-8214-5869cffaa041'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:09.191056 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-aa4722ac-5cc6-42f3-8214-5869cffaa041 admin admin] The token cache is being invalidate because project c5513a83ea324731b26e7223a468f382 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:23:09.191378 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-aa4722ac-5cc6-42f3-8214-5869cffaa041 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:09.192028 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-aa4722ac-5cc6-42f3-8214-5869cffaa041 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:09.193605 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1132/2259] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:23:09 2022] DELETE /identity/v3/projects/c5513a83ea324731b26e7223a468f382 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:09.194971 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f38d80-8e77-4b97-b81c-8a20a867557f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:09.195287 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f38d80-8e77-4b97-b81c-8a20a867557f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:09.195675 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f38d80-8e77-4b97-b81c-8a20a867557f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:09.196679 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cc1a2-60e6-436f-acb5-368879d1becb admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:09.221606 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-51f38d80-8e77-4b97-b81c-8a20a867557f None None] MFA Rules not processed for user `e8e8763ea0a04caea7d81b816c085817`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:09.222810 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-51f38d80-8e77-4b97-b81c-8a20a867557f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:09.258179 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08deb3-00ee-40f8-9bd0-595792eba577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:09.258983 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08deb3-00ee-40f8-9bd0-595792eba577 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:09.259271 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08deb3-00ee-40f8-9bd0-595792eba577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:09.259516 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08deb3-00ee-40f8-9bd0-595792eba577 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:09.260288 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08deb3-00ee-40f8-9bd0-595792eba577 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:09.262138 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08deb3-00ee-40f8-9bd0-595792eba577 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:09.272132 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1128/2260] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:23:09.273362 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f707328-0112-4c3a-9098-e5c499725d14 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:09.273658 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f707328-0112-4c3a-9098-e5c499725d14 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:09.273933 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f707328-0112-4c3a-9098-e5c499725d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:09.290185 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1133/2261] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:09.291103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f38d80-8e77-4b97-b81c-8a20a867557f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:09.294750 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-51f38d80-8e77-4b97-b81c-8a20a867557f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:09.294959 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0f707328-0112-4c3a-9098-e5c499725d14 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:09.296206 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0f707328-0112-4c3a-9098-e5c499725d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:09.307552 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1129/2262] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:09.308405 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f707328-0112-4c3a-9098-e5c499725d14 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:09.310271 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f38d80-8e77-4b97-b81c-8a20a867557f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:09.310391 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0f707328-0112-4c3a-9098-e5c499725d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:09.317456 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df51e72-65cd-42de-a24f-17afabbfebab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:09.318129 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df51e72-65cd-42de-a24f-17afabbfebab admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:09.318411 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df51e72-65cd-42de-a24f-17afabbfebab admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:09.318689 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df51e72-65cd-42de-a24f-17afabbfebab admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:09.319477 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df51e72-65cd-42de-a24f-17afabbfebab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:09.321139 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df51e72-65cd-42de-a24f-17afabbfebab admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:09.331641 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1134/2263] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:23:09.335676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b9b8b-a55d-4889-aeff-fe0ba00b9c0d None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:09.336039 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b9b8b-a55d-4889-aeff-fe0ba00b9c0d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:09.336327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b9b8b-a55d-4889-aeff-fe0ba00b9c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:09.354517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-ea5b9b8b-a55d-4889-aeff-fe0ba00b9c0d None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:09.356283 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ea5b9b8b-a55d-4889-aeff-fe0ba00b9c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:09.364942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1135/2264] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:09.365686 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b9b8b-a55d-4889-aeff-fe0ba00b9c0d None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:09.367730 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f707328-0112-4c3a-9098-e5c499725d14 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:09.371924 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634fbdfa-b14c-4bbb-a23a-418ecf3c823c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:09.372682 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634fbdfa-b14c-4bbb-a23a-418ecf3c823c service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:09.372976 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634fbdfa-b14c-4bbb-a23a-418ecf3c823c service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:09.373253 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634fbdfa-b14c-4bbb-a23a-418ecf3c823c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:09.375981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b9b8b-a55d-4889-aeff-fe0ba00b9c0d None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:09.380652 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe249e8-cb7c-4a41-b693-4e4a948a0100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:09.381533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe249e8-cb7c-4a41-b693-4e4a948a0100 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:09.381818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe249e8-cb7c-4a41-b693-4e4a948a0100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:09.382090 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe249e8-cb7c-4a41-b693-4e4a948a0100 admin admin] PATH_INFO: `/v3/projects/dcaf9afb5d434e7f99d38774f068cbfd/users/47a2da7dc2d849f59c0817f79e339f27/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:09.382190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634fbdfa-b14c-4bbb-a23a-418ecf3c823c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:09.383618 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634fbdfa-b14c-4bbb-a23a-418ecf3c823c service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:09.392772 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1130/2265] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:23:09.393573 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634fbdfa-b14c-4bbb-a23a-418ecf3c823c service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:09.397002 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe249e8-cb7c-4a41-b693-4e4a948a0100 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcaf9afb5d434e7f99d38774f068cbfd, user_id=47a2da7dc2d849f59c0817f79e339f27, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:09.399697 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe249e8-cb7c-4a41-b693-4e4a948a0100 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:09.403895 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634fbdfa-b14c-4bbb-a23a-418ecf3c823c service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:09.408886 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b502ed-7bd8-450e-97aa-0b34f160e9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:09.409538 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b502ed-7bd8-450e-97aa-0b34f160e9d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:09.409874 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b502ed-7bd8-450e-97aa-0b34f160e9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:09.410153 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b502ed-7bd8-450e-97aa-0b34f160e9d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:09.410903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b502ed-7bd8-450e-97aa-0b34f160e9d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:09.411212 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b502ed-7bd8-450e-97aa-0b34f160e9d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:09.412939 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b502ed-7bd8-450e-97aa-0b34f160e9d5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:09.414394 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1136/2266] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:23:09 2022] PUT /identity/v3/projects/dcaf9afb5d434e7f99d38774f068cbfd/users/47a2da7dc2d849f59c0817f79e339f27/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:09.425155 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e62331-d22d-42d7-bf12-885b89a18792 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:09.425499 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e62331-d22d-42d7-bf12-885b89a18792 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:09.425709 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e62331-d22d-42d7-bf12-885b89a18792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:09.426715 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1131/2267] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:09.430548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b502ed-7bd8-450e-97aa-0b34f160e9d5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:09.440994 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b502ed-7bd8-450e-97aa-0b34f160e9d5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:09.454957 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:09.455738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a5e62331-d22d-42d7-bf12-885b89a18792 None None] MFA Rules not processed for user `47a2da7dc2d849f59c0817f79e339f27`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:09.457032 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a5e62331-d22d-42d7-bf12-885b89a18792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:09.513201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a830a1-c048-45c4-ba8d-d0610af80eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:09.515310 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a830a1-c048-45c4-ba8d-d0610af80eca admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:09.515752 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a830a1-c048-45c4-ba8d-d0610af80eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:09.516036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a830a1-c048-45c4-ba8d-d0610af80eca admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:09.517158 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:09.518472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a830a1-c048-45c4-ba8d-d0610af80eca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:09.523956 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a830a1-c048-45c4-ba8d-d0610af80eca admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:09.563385 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1137/2268] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:23:09.566271 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e62331-d22d-42d7-bf12-885b89a18792 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:09.568097 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1132/2269] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:23:09 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:23:09.569892 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a830a1-c048-45c4-ba8d-d0610af80eca admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:09.585570 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e62331-d22d-42d7-bf12-885b89a18792 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:09.666737 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a830a1-c048-45c4-ba8d-d0610af80eca admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:09.675167 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da9f6f-144a-4b47-abe4-3834b07ca87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:09.676735 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da9f6f-144a-4b47-abe4-3834b07ca87b admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:09.677025 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da9f6f-144a-4b47-abe4-3834b07ca87b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:09.677392 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da9f6f-144a-4b47-abe4-3834b07ca87b admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:09.678613 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09da9f6f-144a-4b47-abe4-3834b07ca87b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:09.681519 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09da9f6f-144a-4b47-abe4-3834b07ca87b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:09.700730 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b70db-8159-4e0b-ada2-210d85e0a469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:09.701412 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b70db-8159-4e0b-ada2-210d85e0a469 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:09.701701 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b70db-8159-4e0b-ada2-210d85e0a469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:09.701985 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b70db-8159-4e0b-ada2-210d85e0a469 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:09.702696 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0b70db-8159-4e0b-ada2-210d85e0a469 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:09.704730 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0b70db-8159-4e0b-ada2-210d85e0a469 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:09.710354 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1133/2270] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:23:09 2022] POST /identity/v3/users => generated 287 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:09.711331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da9f6f-144a-4b47-abe4-3834b07ca87b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:09.713286 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-09da9f6f-144a-4b47-abe4-3834b07ca87b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:09.728267 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da9f6f-144a-4b47-abe4-3834b07ca87b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:09.736687 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9fd1f4-7a12-4f1a-9381-ae009c5cf477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:09.737686 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fd1f4-7a12-4f1a-9381-ae009c5cf477 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:09.737971 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fd1f4-7a12-4f1a-9381-ae009c5cf477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:09.738349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fd1f4-7a12-4f1a-9381-ae009c5cf477 admin admin] PATH_INFO: `/v3/users/3d6162c3ea6b4ee196fe2266bddc7044` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:09.740043 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9fd1f4-7a12-4f1a-9381-ae009c5cf477 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d6162c3ea6b4ee196fe2266bddc7044)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:09.742039 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9fd1f4-7a12-4f1a-9381-ae009c5cf477 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:09.766464 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1138/2271] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:23:09 2022] POST /identity/v3/projects => generated 373 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:09.767037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b70db-8159-4e0b-ada2-210d85e0a469 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:09.816540 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cc9fd1f4-7a12-4f1a-9381-ae009c5cf477 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d6162c3ea6b4ee196fe2266bddc7044'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:09.817248 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cc9fd1f4-7a12-4f1a-9381-ae009c5cf477 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d6162c3ea6b4ee196fe2266bddc7044'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:09.817891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cc9fd1f4-7a12-4f1a-9381-ae009c5cf477 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d6162c3ea6b4ee196fe2266bddc7044'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:09.832855 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cc9fd1f4-7a12-4f1a-9381-ae009c5cf477 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d6162c3ea6b4ee196fe2266bddc7044'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:09.855440 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cc9fd1f4-7a12-4f1a-9381-ae009c5cf477 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d6162c3ea6b4ee196fe2266bddc7044'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:09.884973 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1134/2272] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:23:09 2022] DELETE /identity/v3/users/3d6162c3ea6b4ee196fe2266bddc7044 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:09.885841 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9fd1f4-7a12-4f1a-9381-ae009c5cf477 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:09.892214 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cc9fd1f4-7a12-4f1a-9381-ae009c5cf477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:09.900193 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b70db-8159-4e0b-ada2-210d85e0a469 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:09.908804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:09.991711 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b361c0d-4d74-42a3-830b-794cb4646d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:09.992448 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b361c0d-4d74-42a3-830b-794cb4646d91 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:09.992740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b361c0d-4d74-42a3-830b-794cb4646d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:09.993241 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b361c0d-4d74-42a3-830b-794cb4646d91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:09.993793 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b361c0d-4d74-42a3-830b-794cb4646d91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:09.995476 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b361c0d-4d74-42a3-830b-794cb4646d91 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:10.007750 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9fd1f4-7a12-4f1a-9381-ae009c5cf477 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:10.014323 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1139/2273] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:23:10.014323 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa1f45-63f1-4193-b66a-fc4dba2c400b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:10.014323 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa1f45-63f1-4193-b66a-fc4dba2c400b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:10.014323 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa1f45-63f1-4193-b66a-fc4dba2c400b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:10.014714 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9086c1c-5e2e-4a52-a5e0-69c869e61e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:10.014714 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9086c1c-5e2e-4a52-a5e0-69c869e61e9c admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:10.014714 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9086c1c-5e2e-4a52-a5e0-69c869e61e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:10.014714 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9086c1c-5e2e-4a52-a5e0-69c869e61e9c admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:10.015184 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9086c1c-5e2e-4a52-a5e0-69c869e61e9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:10.017207 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9086c1c-5e2e-4a52-a5e0-69c869e61e9c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:10.033718 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-81aa1f45-63f1-4193-b66a-fc4dba2c400b None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:10.034926 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-81aa1f45-63f1-4193-b66a-fc4dba2c400b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:10.038844 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1135/2274] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:23:09 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:10.039674 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9086c1c-5e2e-4a52-a5e0-69c869e61e9c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:10.041442 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a9086c1c-5e2e-4a52-a5e0-69c869e61e9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:10.044207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1140/2275] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:10.045017 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa1f45-63f1-4193-b66a-fc4dba2c400b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:10.049813 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:10.056968 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa1f45-63f1-4193-b66a-fc4dba2c400b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:10.057285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9086c1c-5e2e-4a52-a5e0-69c869e61e9c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:10.062105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a27f6b-9f00-44bc-b0c0-cba037913082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:10.062314 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b849c6-af72-47e9-93c9-731e6079cba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:10.062920 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a27f6b-9f00-44bc-b0c0-cba037913082 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:10.063019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b849c6-af72-47e9-93c9-731e6079cba8 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:10.063276 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b849c6-af72-47e9-93c9-731e6079cba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:10.063426 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a27f6b-9f00-44bc-b0c0-cba037913082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:10.063557 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b849c6-af72-47e9-93c9-731e6079cba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:10.063855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a27f6b-9f00-44bc-b0c0-cba037913082 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:10.064309 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b849c6-af72-47e9-93c9-731e6079cba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:10.064683 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a27f6b-9f00-44bc-b0c0-cba037913082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:10.065898 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b849c6-af72-47e9-93c9-731e6079cba8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:10.066268 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a27f6b-9f00-44bc-b0c0-cba037913082 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:10.076870 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1136/2276] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:23:10.077340 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1141/2277] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:23:10.077693 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b849c6-af72-47e9-93c9-731e6079cba8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:10.079377 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-33b849c6-af72-47e9-93c9-731e6079cba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:10.080970 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a27f6b-9f00-44bc-b0c0-cba037913082 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:10.092704 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a27f6b-9f00-44bc-b0c0-cba037913082 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:10.097472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278126b-8844-413c-85ab-dde6873e0e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:10.098376 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278126b-8844-413c-85ab-dde6873e0e6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:10.098659 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278126b-8844-413c-85ab-dde6873e0e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:10.099196 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278126b-8844-413c-85ab-dde6873e0e6a admin admin] PATH_INFO: `/v3/projects/ecb2b7cbdaf9495480fc881e823c1913/users/becdcd1df4a641e6a5bda05220f7aa07/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:10.114151 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5278126b-8844-413c-85ab-dde6873e0e6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecb2b7cbdaf9495480fc881e823c1913, user_id=becdcd1df4a641e6a5bda05220f7aa07, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:10.116843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5278126b-8844-413c-85ab-dde6873e0e6a admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:10.130285 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1142/2278] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:23:10 2022] PUT /identity/v3/projects/ecb2b7cbdaf9495480fc881e823c1913/users/becdcd1df4a641e6a5bda05220f7aa07/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:23:10.134243 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278126b-8844-413c-85ab-dde6873e0e6a admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:10.140415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b849c6-af72-47e9-93c9-731e6079cba8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:10.195516 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b3e8c-a3de-4603-976e-295925306236 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:10.196173 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b3e8c-a3de-4603-976e-295925306236 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:10.196456 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b3e8c-a3de-4603-976e-295925306236 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:10.196729 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b3e8c-a3de-4603-976e-295925306236 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:10.198461 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278126b-8844-413c-85ab-dde6873e0e6a admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:10.203350 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03dbdf-c359-47b2-97cd-0f74926a01a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:10.204237 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03dbdf-c359-47b2-97cd-0f74926a01a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:10.204563 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03dbdf-c359-47b2-97cd-0f74926a01a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:10.204839 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03dbdf-c359-47b2-97cd-0f74926a01a6 admin admin] PATH_INFO: `/v3/projects/18290df1b4ee407c9cabdbaaedc1f64b/users/40cf147c05d241cc9341eda110a28822/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:10.206244 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b3e8c-a3de-4603-976e-295925306236 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:10.207955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b3e8c-a3de-4603-976e-295925306236 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:10.218800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1137/2279] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:23:10.220228 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e72ee3-6230-4f05-bfff-5c6766abbf8b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:10.220535 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e72ee3-6230-4f05-bfff-5c6766abbf8b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:10.220794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e72ee3-6230-4f05-bfff-5c6766abbf8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:10.220992 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd03dbdf-c359-47b2-97cd-0f74926a01a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18290df1b4ee407c9cabdbaaedc1f64b, user_id=40cf147c05d241cc9341eda110a28822, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:10.225491 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd03dbdf-c359-47b2-97cd-0f74926a01a6 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:10.239896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-74e72ee3-6230-4f05-bfff-5c6766abbf8b None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:10.241251 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-74e72ee3-6230-4f05-bfff-5c6766abbf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:10.242322 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1143/2280] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:23:10 2022] PUT /identity/v3/projects/18290df1b4ee407c9cabdbaaedc1f64b/users/40cf147c05d241cc9341eda110a28822/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:10.247409 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03dbdf-c359-47b2-97cd-0f74926a01a6 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:10.288984 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:10.319917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1138/2281] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:10.331174 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03dbdf-c359-47b2-97cd-0f74926a01a6 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:10.331330 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e72ee3-6230-4f05-bfff-5c6766abbf8b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:10.338397 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbb5c4-7e72-414b-9e4f-4627176f7a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:10.339133 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbb5c4-7e72-414b-9e4f-4627176f7a13 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:10.339492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbb5c4-7e72-414b-9e4f-4627176f7a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:10.339881 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbb5c4-7e72-414b-9e4f-4627176f7a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:10.340906 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cbb5c4-7e72-414b-9e4f-4627176f7a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:10.342698 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cbb5c4-7e72-414b-9e4f-4627176f7a13 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:10.356240 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1144/2282] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:23:10.360159 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbb5c4-7e72-414b-9e4f-4627176f7a13 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:10.373009 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbb5c4-7e72-414b-9e4f-4627176f7a13 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:10.378218 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad22fe-a785-4912-a06b-33312c3e93d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:10.379562 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad22fe-a785-4912-a06b-33312c3e93d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:10.379562 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad22fe-a785-4912-a06b-33312c3e93d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:10.379778 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad22fe-a785-4912-a06b-33312c3e93d6 admin admin] PATH_INFO: `/v3/projects/18290df1b4ee407c9cabdbaaedc1f64b/users/40cf147c05d241cc9341eda110a28822/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:10.381974 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ad22fe-a785-4912-a06b-33312c3e93d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18290df1b4ee407c9cabdbaaedc1f64b, user_id=40cf147c05d241cc9341eda110a28822, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:10.385602 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ad22fe-a785-4912-a06b-33312c3e93d6 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:10.399551 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1145/2283] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:23:10 2022] PUT /identity/v3/projects/18290df1b4ee407c9cabdbaaedc1f64b/users/40cf147c05d241cc9341eda110a28822/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:23:10.410948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e72ee3-6230-4f05-bfff-5c6766abbf8b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:10.412448 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8ba3f-4631-44c4-983c-a7bf0890ee71 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:10.412741 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8ba3f-4631-44c4-983c-a7bf0890ee71 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:10.413257 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8ba3f-4631-44c4-983c-a7bf0890ee71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:10.450955 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-e9d8ba3f-4631-44c4-983c-a7bf0890ee71 None None] MFA Rules not processed for user `40cf147c05d241cc9341eda110a28822`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:10.452227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e9d8ba3f-4631-44c4-983c-a7bf0890ee71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:10.477051 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:10.479985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb2480-e683-4c13-96da-49a0ccfe4dbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:10.481573 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb2480-e683-4c13-96da-49a0ccfe4dbf service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:10.481864 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb2480-e683-4c13-96da-49a0ccfe4dbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:10.482243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb2480-e683-4c13-96da-49a0ccfe4dbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:10.541135 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1146/2284] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:10 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:23:10.555123 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d8ba3f-4631-44c4-983c-a7bf0890ee71 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:10.557072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e9d8ba3f-4631-44c4-983c-a7bf0890ee71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:10.568242 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb2480-e683-4c13-96da-49a0ccfe4dbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:10.569675 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb2480-e683-4c13-96da-49a0ccfe4dbf service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:10.584572 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1139/2285] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:23:10.585454 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb2480-e683-4c13-96da-49a0ccfe4dbf service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:10.586895 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fccb2480-e683-4c13-96da-49a0ccfe4dbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:10.641078 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d8ba3f-4631-44c4-983c-a7bf0890ee71 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:10.647035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d00a3-b124-408d-b2c7-2da6fab42b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:10.648138 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d00a3-b124-408d-b2c7-2da6fab42b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:10.648355 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d00a3-b124-408d-b2c7-2da6fab42b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:10.648781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d00a3-b124-408d-b2c7-2da6fab42b3c admin admin] PATH_INFO: `/v3/projects/a087eb5b34f04e51ad8462cc4deab158` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:10.649942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d00a3-b124-408d-b2c7-2da6fab42b3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a087eb5b34f04e51ad8462cc4deab158)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:10.652733 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d00a3-b124-408d-b2c7-2da6fab42b3c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:10.654411 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:10.658863 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb2480-e683-4c13-96da-49a0ccfe4dbf service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:10.663292 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f41933-a05e-4ff3-a617-8b92f250a0fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:10.664247 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f41933-a05e-4ff3-a617-8b92f250a0fd service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:10.664417 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f41933-a05e-4ff3-a617-8b92f250a0fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:10.665039 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f41933-a05e-4ff3-a617-8b92f250a0fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:10.666989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-46f41933-a05e-4ff3-a617-8b92f250a0fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:10.711308 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-209d00a3-b124-408d-b2c7-2da6fab42b3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a087eb5b34f04e51ad8462cc4deab158', 'request_id': 'req-209d00a3-b124-408d-b2c7-2da6fab42b3c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:10.720775 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-209d00a3-b124-408d-b2c7-2da6fab42b3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a087eb5b34f04e51ad8462cc4deab158', 'request_id': 'req-209d00a3-b124-408d-b2c7-2da6fab42b3c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:10.728411 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-209d00a3-b124-408d-b2c7-2da6fab42b3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a087eb5b34f04e51ad8462cc4deab158', 'request_id': 'req-209d00a3-b124-408d-b2c7-2da6fab42b3c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:10.736845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f41933-a05e-4ff3-a617-8b92f250a0fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:10.738497 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f41933-a05e-4ff3-a617-8b92f250a0fd service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:10.742583 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-209d00a3-b124-408d-b2c7-2da6fab42b3c admin admin] The token cache is being invalidate because project a087eb5b34f04e51ad8462cc4deab158 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:23:10.742953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-209d00a3-b124-408d-b2c7-2da6fab42b3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:10.744073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-209d00a3-b124-408d-b2c7-2da6fab42b3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:10.747068 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1147/2286] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:23:10 2022] DELETE /identity/v3/projects/a087eb5b34f04e51ad8462cc4deab158 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:10.748425 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1140/2287] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:23:10.750729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d00a3-b124-408d-b2c7-2da6fab42b3c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:10.752240 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-209d00a3-b124-408d-b2c7-2da6fab42b3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:10.817991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:10.844420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d00a3-b124-408d-b2c7-2da6fab42b3c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:10.850306 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593894cb-e423-4da1-8e2d-c978577fd92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:10.851547 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593894cb-e423-4da1-8e2d-c978577fd92d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:10.851836 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593894cb-e423-4da1-8e2d-c978577fd92d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:10.852118 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593894cb-e423-4da1-8e2d-c978577fd92d admin admin] PATH_INFO: `/v3/users/8f17057c890d448290fbd1ed76205351` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:10.853713 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593894cb-e423-4da1-8e2d-c978577fd92d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f17057c890d448290fbd1ed76205351)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:10.855950 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593894cb-e423-4da1-8e2d-c978577fd92d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:10.928029 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-593894cb-e423-4da1-8e2d-c978577fd92d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f17057c890d448290fbd1ed76205351'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:10.942520 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-593894cb-e423-4da1-8e2d-c978577fd92d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f17057c890d448290fbd1ed76205351'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:10.943565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-593894cb-e423-4da1-8e2d-c978577fd92d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f17057c890d448290fbd1ed76205351'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:10.944483 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-593894cb-e423-4da1-8e2d-c978577fd92d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f17057c890d448290fbd1ed76205351'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:10.953853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-593894cb-e423-4da1-8e2d-c978577fd92d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f17057c890d448290fbd1ed76205351'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:10.974594 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1148/2288] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:23:10 2022] DELETE /identity/v3/users/8f17057c890d448290fbd1ed76205351 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:23:11.075462 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f41933-a05e-4ff3-a617-8b92f250a0fd service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:11.080069 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-46f41933-a05e-4ff3-a617-8b92f250a0fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:11.105778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f41933-a05e-4ff3-a617-8b92f250a0fd service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:11.231804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e468d-e1df-45f3-af96-144414694b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:11.233081 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e468d-e1df-45f3-af96-144414694b19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:11.233450 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e468d-e1df-45f3-af96-144414694b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:11.234073 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e468d-e1df-45f3-af96-144414694b19 admin admin] PATH_INFO: `/v3/users/e7db45e6d45a47baabf130d4e039681f` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:11.236544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e468d-e1df-45f3-af96-144414694b19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7db45e6d45a47baabf130d4e039681f)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:11.239751 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e468d-e1df-45f3-af96-144414694b19 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:11.310334 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-db4e468d-e1df-45f3-af96-144414694b19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7db45e6d45a47baabf130d4e039681f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:11.311185 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-db4e468d-e1df-45f3-af96-144414694b19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7db45e6d45a47baabf130d4e039681f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:11.312008 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-db4e468d-e1df-45f3-af96-144414694b19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7db45e6d45a47baabf130d4e039681f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:11.328460 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-db4e468d-e1df-45f3-af96-144414694b19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7db45e6d45a47baabf130d4e039681f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:11.356445 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-db4e468d-e1df-45f3-af96-144414694b19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7db45e6d45a47baabf130d4e039681f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:11.379506 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1141/2289] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:23:11 2022] DELETE /identity/v3/users/e7db45e6d45a47baabf130d4e039681f => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:11.384708 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75433428-7118-4d26-8012-9eda3e3cb0f0 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:11.385216 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75433428-7118-4d26-8012-9eda3e3cb0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:11.385702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75433428-7118-4d26-8012-9eda3e3cb0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:11.449616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-75433428-7118-4d26-8012-9eda3e3cb0f0 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:11.451041 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-75433428-7118-4d26-8012-9eda3e3cb0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:11.525547 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c2e2a-2734-4e30-87ad-3941c4cc6203 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:11.526609 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c2e2a-2734-4e30-87ad-3941c4cc6203 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:11.526609 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c2e2a-2734-4e30-87ad-3941c4cc6203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:11.582776 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-016c2e2a-2734-4e30-87ad-3941c4cc6203 None None] MFA Rules not processed for user `becdcd1df4a641e6a5bda05220f7aa07`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:11.584584 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-016c2e2a-2734-4e30-87ad-3941c4cc6203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:11.599345 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:11.619656 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1149/2290] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:11.620428 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75433428-7118-4d26-8012-9eda3e3cb0f0 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:11.622859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-75433428-7118-4d26-8012-9eda3e3cb0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:11.645269 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75433428-7118-4d26-8012-9eda3e3cb0f0 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:11.656082 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad32c6-0eb0-4576-ba0c-70cada94c29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:11.657245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad32c6-0eb0-4576-ba0c-70cada94c29d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:11.657594 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad32c6-0eb0-4576-ba0c-70cada94c29d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:11.657897 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad32c6-0eb0-4576-ba0c-70cada94c29d admin admin] PATH_INFO: `/v3/projects/5afcaee98c1341cba05a9ac50a0bc4ec` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:11.659472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad32c6-0eb0-4576-ba0c-70cada94c29d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5afcaee98c1341cba05a9ac50a0bc4ec)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:11.662806 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad32c6-0eb0-4576-ba0c-70cada94c29d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:11.709528 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1142/2291] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:23:11.710432 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c2e2a-2734-4e30-87ad-3941c4cc6203 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:11.713119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-016c2e2a-2734-4e30-87ad-3941c4cc6203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:11.737796 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-14ad32c6-0eb0-4576-ba0c-70cada94c29d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5afcaee98c1341cba05a9ac50a0bc4ec', 'request_id': 'req-14ad32c6-0eb0-4576-ba0c-70cada94c29d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:11.746770 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-14ad32c6-0eb0-4576-ba0c-70cada94c29d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5afcaee98c1341cba05a9ac50a0bc4ec', 'request_id': 'req-14ad32c6-0eb0-4576-ba0c-70cada94c29d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:11.754701 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-14ad32c6-0eb0-4576-ba0c-70cada94c29d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5afcaee98c1341cba05a9ac50a0bc4ec', 'request_id': 'req-14ad32c6-0eb0-4576-ba0c-70cada94c29d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:11.757844 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:11.769739 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-14ad32c6-0eb0-4576-ba0c-70cada94c29d admin admin] The token cache is being invalidate because project 5afcaee98c1341cba05a9ac50a0bc4ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:23:11.770322 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-14ad32c6-0eb0-4576-ba0c-70cada94c29d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:11.771202 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-14ad32c6-0eb0-4576-ba0c-70cada94c29d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:11.772826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1150/2292] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:23:11 2022] DELETE /identity/v3/projects/5afcaee98c1341cba05a9ac50a0bc4ec => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:11.776009 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad32c6-0eb0-4576-ba0c-70cada94c29d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:11.778140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-14ad32c6-0eb0-4576-ba0c-70cada94c29d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:11.787588 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c2e2a-2734-4e30-87ad-3941c4cc6203 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:11.837025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad32c6-0eb0-4576-ba0c-70cada94c29d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:11.840679 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95948e0f-7495-486e-8b0e-1ee88b8b03e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:11.841471 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4adb499-3f55-434a-a0ea-4412ced9d75e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:11.841639 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95948e0f-7495-486e-8b0e-1ee88b8b03e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:11.841738 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95948e0f-7495-486e-8b0e-1ee88b8b03e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:11.842007 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95948e0f-7495-486e-8b0e-1ee88b8b03e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:11.842262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4adb499-3f55-434a-a0ea-4412ced9d75e service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:11.842550 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4adb499-3f55-434a-a0ea-4412ced9d75e service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:11.842882 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4adb499-3f55-434a-a0ea-4412ced9d75e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:11.843925 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-95948e0f-7495-486e-8b0e-1ee88b8b03e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:11.844759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f4adb499-3f55-434a-a0ea-4412ced9d75e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:11.905972 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4adb499-3f55-434a-a0ea-4412ced9d75e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:11.907702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4adb499-3f55-434a-a0ea-4412ced9d75e service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:11.908456 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95948e0f-7495-486e-8b0e-1ee88b8b03e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:11.909884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95948e0f-7495-486e-8b0e-1ee88b8b03e9 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:11.916609 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1151/2293] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:23:11.918259 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4adb499-3f55-434a-a0ea-4412ced9d75e service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:11.919539 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1143/2294] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:23:11.919861 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f4adb499-3f55-434a-a0ea-4412ced9d75e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:11.934436 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4adb499-3f55-434a-a0ea-4412ced9d75e service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:11.940043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414d583-6ab1-404a-91c6-9e1d30c5855d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:11.940914 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414d583-6ab1-404a-91c6-9e1d30c5855d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:11.941199 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414d583-6ab1-404a-91c6-9e1d30c5855d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:11.941477 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414d583-6ab1-404a-91c6-9e1d30c5855d admin admin] PATH_INFO: `/v3/users/4e2740c5cfba43dbae569b72d72b6847` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:11.942687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9414d583-6ab1-404a-91c6-9e1d30c5855d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e2740c5cfba43dbae569b72d72b6847)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:11.944629 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9414d583-6ab1-404a-91c6-9e1d30c5855d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:12.006575 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9414d583-6ab1-404a-91c6-9e1d30c5855d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e2740c5cfba43dbae569b72d72b6847'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:12.023061 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9414d583-6ab1-404a-91c6-9e1d30c5855d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e2740c5cfba43dbae569b72d72b6847'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:12.023948 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9414d583-6ab1-404a-91c6-9e1d30c5855d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e2740c5cfba43dbae569b72d72b6847'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:12.024615 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9414d583-6ab1-404a-91c6-9e1d30c5855d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e2740c5cfba43dbae569b72d72b6847'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:12.033494 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9414d583-6ab1-404a-91c6-9e1d30c5855d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e2740c5cfba43dbae569b72d72b6847'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:12.047870 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1152/2295] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:23:11 2022] DELETE /identity/v3/users/4e2740c5cfba43dbae569b72d72b6847 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:12.345296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95948e0f-7495-486e-8b0e-1ee88b8b03e9 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:12.347291 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-95948e0f-7495-486e-8b0e-1ee88b8b03e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:12.365865 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95948e0f-7495-486e-8b0e-1ee88b8b03e9 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:12.384865 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:12.431047 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9e277-c586-47a6-b7a1-ac23cf51aea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:12.431943 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9e277-c586-47a6-b7a1-ac23cf51aea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:12.432224 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9e277-c586-47a6-b7a1-ac23cf51aea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:12.432496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9e277-c586-47a6-b7a1-ac23cf51aea9 admin admin] PATH_INFO: `/v3/projects/a24f5f4930bf4726910dd1c12dadc83a` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:12.434267 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9e277-c586-47a6-b7a1-ac23cf51aea9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a24f5f4930bf4726910dd1c12dadc83a)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:12.436381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9e277-c586-47a6-b7a1-ac23cf51aea9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:12.452986 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414d583-6ab1-404a-91c6-9e1d30c5855d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:12.454669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9414d583-6ab1-404a-91c6-9e1d30c5855d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:12.477834 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414d583-6ab1-404a-91c6-9e1d30c5855d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:12.486294 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea74c8d3-d3aa-4601-83f0-a0a2c8b2f2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:12.487687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74c8d3-d3aa-4601-83f0-a0a2c8b2f2d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:12.487979 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74c8d3-d3aa-4601-83f0-a0a2c8b2f2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:12.488357 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74c8d3-d3aa-4601-83f0-a0a2c8b2f2d1 admin admin] PATH_INFO: `/v3/projects/82ef8fdeaf224ff9bd8f84c7ff9c31cc` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:12.489599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74c8d3-d3aa-4601-83f0-a0a2c8b2f2d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82ef8fdeaf224ff9bd8f84c7ff9c31cc)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:12.492333 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74c8d3-d3aa-4601-83f0-a0a2c8b2f2d1 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:12.505352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c2f9e277-c586-47a6-b7a1-ac23cf51aea9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a24f5f4930bf4726910dd1c12dadc83a', 'request_id': 'req-c2f9e277-c586-47a6-b7a1-ac23cf51aea9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:12.506063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:12.516316 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c2f9e277-c586-47a6-b7a1-ac23cf51aea9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a24f5f4930bf4726910dd1c12dadc83a', 'request_id': 'req-c2f9e277-c586-47a6-b7a1-ac23cf51aea9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:12.523318 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c2f9e277-c586-47a6-b7a1-ac23cf51aea9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a24f5f4930bf4726910dd1c12dadc83a', 'request_id': 'req-c2f9e277-c586-47a6-b7a1-ac23cf51aea9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:12.535784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c2f9e277-c586-47a6-b7a1-ac23cf51aea9 admin admin] The token cache is being invalidate because project a24f5f4930bf4726910dd1c12dadc83a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:23:12.536081 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c2f9e277-c586-47a6-b7a1-ac23cf51aea9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:12.536728 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c2f9e277-c586-47a6-b7a1-ac23cf51aea9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:12.538776 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1144/2296] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:23:12 2022] DELETE /identity/v3/projects/a24f5f4930bf4726910dd1c12dadc83a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:12.544785 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658af780-934a-4769-adcb-cf5a9fe44fc6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:12.545075 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658af780-934a-4769-adcb-cf5a9fe44fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:12.545352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658af780-934a-4769-adcb-cf5a9fe44fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:12.548022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ea74c8d3-d3aa-4601-83f0-a0a2c8b2f2d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82ef8fdeaf224ff9bd8f84c7ff9c31cc', 'request_id': 'req-ea74c8d3-d3aa-4601-83f0-a0a2c8b2f2d1'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:12.557472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ea74c8d3-d3aa-4601-83f0-a0a2c8b2f2d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82ef8fdeaf224ff9bd8f84c7ff9c31cc', 'request_id': 'req-ea74c8d3-d3aa-4601-83f0-a0a2c8b2f2d1'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:12.565902 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-658af780-934a-4769-adcb-cf5a9fe44fc6 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:12.566147 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ea74c8d3-d3aa-4601-83f0-a0a2c8b2f2d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82ef8fdeaf224ff9bd8f84c7ff9c31cc', 'request_id': 'req-ea74c8d3-d3aa-4601-83f0-a0a2c8b2f2d1'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:12.567284 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-658af780-934a-4769-adcb-cf5a9fe44fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:12.581443 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ea74c8d3-d3aa-4601-83f0-a0a2c8b2f2d1 admin admin] The token cache is being invalidate because project 82ef8fdeaf224ff9bd8f84c7ff9c31cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:23:12.582912 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ea74c8d3-d3aa-4601-83f0-a0a2c8b2f2d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:12.587637 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ea74c8d3-d3aa-4601-83f0-a0a2c8b2f2d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:12.590867 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1153/2297] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:23:12 2022] DELETE /identity/v3/projects/82ef8fdeaf224ff9bd8f84c7ff9c31cc => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:12.662611 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1145/2298] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:12.671752 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041dd2c-117b-4e5e-9360-e3b4076b8044 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:12.672290 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041dd2c-117b-4e5e-9360-e3b4076b8044 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:12.672380 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041dd2c-117b-4e5e-9360-e3b4076b8044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:12.691803 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a041dd2c-117b-4e5e-9360-e3b4076b8044 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:12.693054 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a041dd2c-117b-4e5e-9360-e3b4076b8044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:12.701562 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1154/2299] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:23:12.705926 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658af780-934a-4769-adcb-cf5a9fe44fc6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:12.718371 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658af780-934a-4769-adcb-cf5a9fe44fc6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:12.724326 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c64f0-708f-45a1-aa03-13006c7be340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:12.725020 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c64f0-708f-45a1-aa03-13006c7be340 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:12.725304 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c64f0-708f-45a1-aa03-13006c7be340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:12.725579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c64f0-708f-45a1-aa03-13006c7be340 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:12.726325 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c64f0-708f-45a1-aa03-13006c7be340 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:12.726633 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c64f0-708f-45a1-aa03-13006c7be340 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:12.728446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c64f0-708f-45a1-aa03-13006c7be340 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:12.742903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1146/2300] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:12.748115 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041dd2c-117b-4e5e-9360-e3b4076b8044 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:12.758437 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041dd2c-117b-4e5e-9360-e3b4076b8044 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:12.763450 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c852de-ccef-403f-8269-417607939e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:12.764131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c852de-ccef-403f-8269-417607939e7d admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:12.764424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c852de-ccef-403f-8269-417607939e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:12.764731 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c852de-ccef-403f-8269-417607939e7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:12.765437 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c852de-ccef-403f-8269-417607939e7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:12.767405 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c852de-ccef-403f-8269-417607939e7d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:12.800686 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1155/2301] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:23:12 2022] POST /identity/v3/projects => generated 397 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:12.802942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c64f0-708f-45a1-aa03-13006c7be340 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:12.813994 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c64f0-708f-45a1-aa03-13006c7be340 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:12.848132 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:12.877927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb614105-dec0-4cb1-89f9-cddee15adda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:12.878605 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb614105-dec0-4cb1-89f9-cddee15adda3 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:12.878897 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb614105-dec0-4cb1-89f9-cddee15adda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:12.879164 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb614105-dec0-4cb1-89f9-cddee15adda3 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:12.879899 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb614105-dec0-4cb1-89f9-cddee15adda3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:12.882181 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb614105-dec0-4cb1-89f9-cddee15adda3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:12.904614 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1147/2302] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:23:12 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:12.908719 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c852de-ccef-403f-8269-417607939e7d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:12.919792 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c852de-ccef-403f-8269-417607939e7d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:12.924528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0fe8c-f564-4f5c-b523-a170ebf10dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:12.925196 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0fe8c-f564-4f5c-b523-a170ebf10dbb admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:12.925482 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0fe8c-f564-4f5c-b523-a170ebf10dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:12.925762 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0fe8c-f564-4f5c-b523-a170ebf10dbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:12.926494 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0fe8c-f564-4f5c-b523-a170ebf10dbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:12.928210 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0fe8c-f564-4f5c-b523-a170ebf10dbb admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:12.939552 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1156/2303] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:23:12.943771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb614105-dec0-4cb1-89f9-cddee15adda3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:12.954431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb614105-dec0-4cb1-89f9-cddee15adda3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:12.959649 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8767b7-fb55-4333-baf7-ab6087511fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:12.960323 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8767b7-fb55-4333-baf7-ab6087511fee admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:12.960615 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8767b7-fb55-4333-baf7-ab6087511fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:12.960907 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8767b7-fb55-4333-baf7-ab6087511fee admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:12.961682 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8767b7-fb55-4333-baf7-ab6087511fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:12.963302 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8767b7-fb55-4333-baf7-ab6087511fee admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:12.979613 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1148/2304] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:23:12.981662 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0fe8c-f564-4f5c-b523-a170ebf10dbb admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:12.992926 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0fe8c-f564-4f5c-b523-a170ebf10dbb admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:12.998985 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978958b6-b7ce-42a9-82d2-b81a440d1c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:12.999988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978958b6-b7ce-42a9-82d2-b81a440d1c43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:13.002703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978958b6-b7ce-42a9-82d2-b81a440d1c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:13.002703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978958b6-b7ce-42a9-82d2-b81a440d1c43 admin admin] PATH_INFO: `/v3/projects/86d54923b36b4050a1d72199d695eded/users/dea387889c8549df83cb7fc5d2a8f574/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:13.022289 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978958b6-b7ce-42a9-82d2-b81a440d1c43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86d54923b36b4050a1d72199d695eded, user_id=dea387889c8549df83cb7fc5d2a8f574, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:13.027151 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978958b6-b7ce-42a9-82d2-b81a440d1c43 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:13.050393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1157/2305] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:23:12 2022] PUT /identity/v3/projects/86d54923b36b4050a1d72199d695eded/users/dea387889c8549df83cb7fc5d2a8f574/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:23:13.061734 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfb0b1-83cd-44f5-b697-29d36f93e1dc None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:13.062032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfb0b1-83cd-44f5-b697-29d36f93e1dc None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:13.062312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfb0b1-83cd-44f5-b697-29d36f93e1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:13.095853 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-92cfb0b1-83cd-44f5-b697-29d36f93e1dc None None] MFA Rules not processed for user `dea387889c8549df83cb7fc5d2a8f574`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:13.097045 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-92cfb0b1-83cd-44f5-b697-29d36f93e1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:13.164042 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1149/2306] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:13.176213 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978958b6-b7ce-42a9-82d2-b81a440d1c43 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:13.177904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-978958b6-b7ce-42a9-82d2-b81a440d1c43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:13.234665 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978958b6-b7ce-42a9-82d2-b81a440d1c43 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:13.238596 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f284205-279a-49ed-969f-e53fcd5785d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:13.239261 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f284205-279a-49ed-969f-e53fcd5785d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:13.239551 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f284205-279a-49ed-969f-e53fcd5785d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:13.239898 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f284205-279a-49ed-969f-e53fcd5785d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:13.250440 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f284205-279a-49ed-969f-e53fcd5785d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:13.252037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f284205-279a-49ed-969f-e53fcd5785d8 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:13.261585 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1158/2307] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:13 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:23:15.390748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9b924-415a-4a70-829b-feb25c538b1b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:15.391312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9b924-415a-4a70-829b-feb25c538b1b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:15.391312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9b924-415a-4a70-829b-feb25c538b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:15.409696 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5ef9b924-415a-4a70-829b-feb25c538b1b None None] MFA Rules not processed for user `becdcd1df4a641e6a5bda05220f7aa07`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:15.410882 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5ef9b924-415a-4a70-829b-feb25c538b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:15.500128 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1150/2308] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:15.711828 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f284205-279a-49ed-969f-e53fcd5785d8 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:15.713456 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0f284205-279a-49ed-969f-e53fcd5785d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:15.726555 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f284205-279a-49ed-969f-e53fcd5785d8 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:15.730768 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abf893c-beec-4bbf-9559-6925ec5cabab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:15.731452 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf893c-beec-4bbf-9559-6925ec5cabab service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:15.731730 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf893c-beec-4bbf-9559-6925ec5cabab service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:15.732372 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf893c-beec-4bbf-9559-6925ec5cabab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:15.733759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6abf893c-beec-4bbf-9559-6925ec5cabab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:15.786486 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:15.819014 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abf893c-beec-4bbf-9559-6925ec5cabab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:15.820473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abf893c-beec-4bbf-9559-6925ec5cabab service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:15.831676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1159/2309] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:23:16.552015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9b924-415a-4a70-829b-feb25c538b1b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:16.562793 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9b924-415a-4a70-829b-feb25c538b1b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:16.566559 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2112269f-f763-45af-a168-bf311c7ce9b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:16.567207 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112269f-f763-45af-a168-bf311c7ce9b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:16.567526 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112269f-f763-45af-a168-bf311c7ce9b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:16.567802 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112269f-f763-45af-a168-bf311c7ce9b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:16.569609 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2112269f-f763-45af-a168-bf311c7ce9b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:16.625058 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:16.647106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2112269f-f763-45af-a168-bf311c7ce9b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:16.648586 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2112269f-f763-45af-a168-bf311c7ce9b3 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:16.660928 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1151/2310] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:16 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:23:20.760440 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abf893c-beec-4bbf-9559-6925ec5cabab service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:20.762662 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6abf893c-beec-4bbf-9559-6925ec5cabab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:20.891702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abf893c-beec-4bbf-9559-6925ec5cabab service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:20.898275 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6905be-6956-447a-9707-6c204105b6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:20.899653 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6905be-6956-447a-9707-6c204105b6a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:20.900063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6905be-6956-447a-9707-6c204105b6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:20.900350 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6905be-6956-447a-9707-6c204105b6a1 admin admin] PATH_INFO: `/v3/users/e8e8763ea0a04caea7d81b816c085817` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:20.902179 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6905be-6956-447a-9707-6c204105b6a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8e8763ea0a04caea7d81b816c085817)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:20.905104 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6905be-6956-447a-9707-6c204105b6a1 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:20.954848 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ff6905be-6956-447a-9707-6c204105b6a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8e8763ea0a04caea7d81b816c085817'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:20.974062 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ff6905be-6956-447a-9707-6c204105b6a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8e8763ea0a04caea7d81b816c085817'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:20.975045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ff6905be-6956-447a-9707-6c204105b6a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8e8763ea0a04caea7d81b816c085817'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:20.975873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ff6905be-6956-447a-9707-6c204105b6a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8e8763ea0a04caea7d81b816c085817'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:20.983230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ff6905be-6956-447a-9707-6c204105b6a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8e8763ea0a04caea7d81b816c085817'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:20.988525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:21.001645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1160/2311] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:23:20 2022] DELETE /identity/v3/users/e8e8763ea0a04caea7d81b816c085817 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:21.005940 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d739099-54f8-4ee2-9de4-0dbd5466830c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:21.006410 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d739099-54f8-4ee2-9de4-0dbd5466830c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:21.006558 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d739099-54f8-4ee2-9de4-0dbd5466830c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:21.067050 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7d739099-54f8-4ee2-9de4-0dbd5466830c None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:21.068836 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7d739099-54f8-4ee2-9de4-0dbd5466830c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:21.158503 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1152/2312] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:21.173175 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6905be-6956-447a-9707-6c204105b6a1 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:21.174796 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ff6905be-6956-447a-9707-6c204105b6a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:21.288907 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6905be-6956-447a-9707-6c204105b6a1 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:21.293378 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079be2f-4b34-4c06-84df-c526eef6ec08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:21.294232 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079be2f-4b34-4c06-84df-c526eef6ec08 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:21.294516 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079be2f-4b34-4c06-84df-c526eef6ec08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:21.294795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079be2f-4b34-4c06-84df-c526eef6ec08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:21.306307 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079be2f-4b34-4c06-84df-c526eef6ec08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:21.307798 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079be2f-4b34-4c06-84df-c526eef6ec08 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:21.318750 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1161/2313] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:23:21.813759 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d739099-54f8-4ee2-9de4-0dbd5466830c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:21.815696 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7d739099-54f8-4ee2-9de4-0dbd5466830c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:21.838564 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d739099-54f8-4ee2-9de4-0dbd5466830c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:21.838564 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde8f77-1324-4ba6-8a51-6083607a438b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:21.839735 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde8f77-1324-4ba6-8a51-6083607a438b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:21.840143 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde8f77-1324-4ba6-8a51-6083607a438b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:21.840492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde8f77-1324-4ba6-8a51-6083607a438b admin admin] PATH_INFO: `/v3/projects/ae91adf4b6f341a194bca5f0cad50437` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:21.841958 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfde8f77-1324-4ba6-8a51-6083607a438b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae91adf4b6f341a194bca5f0cad50437)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:21.844060 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfde8f77-1324-4ba6-8a51-6083607a438b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:21.905200 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cfde8f77-1324-4ba6-8a51-6083607a438b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae91adf4b6f341a194bca5f0cad50437', 'request_id': 'req-cfde8f77-1324-4ba6-8a51-6083607a438b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:21.916918 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cfde8f77-1324-4ba6-8a51-6083607a438b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae91adf4b6f341a194bca5f0cad50437', 'request_id': 'req-cfde8f77-1324-4ba6-8a51-6083607a438b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:21.927091 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cfde8f77-1324-4ba6-8a51-6083607a438b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae91adf4b6f341a194bca5f0cad50437', 'request_id': 'req-cfde8f77-1324-4ba6-8a51-6083607a438b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:21.939848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cfde8f77-1324-4ba6-8a51-6083607a438b admin admin] The token cache is being invalidate because project ae91adf4b6f341a194bca5f0cad50437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:23:21.940144 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cfde8f77-1324-4ba6-8a51-6083607a438b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:21.940812 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cfde8f77-1324-4ba6-8a51-6083607a438b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:21.947612 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1153/2314] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:23:21 2022] DELETE /identity/v3/projects/ae91adf4b6f341a194bca5f0cad50437 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:21.954348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d0ce1-49b0-443a-aae3-970ffee2a251 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:21.955431 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d0ce1-49b0-443a-aae3-970ffee2a251 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:21.956246 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d0ce1-49b0-443a-aae3-970ffee2a251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:22.007933 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-540d0ce1-49b0-443a-aae3-970ffee2a251 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:22.009182 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-540d0ce1-49b0-443a-aae3-970ffee2a251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:22.085785 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1162/2315] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:23:22.094186 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d4ff0-5459-47f7-ab59-de5c18447639 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:22.094674 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d4ff0-5459-47f7-ab59-de5c18447639 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:22.094763 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d4ff0-5459-47f7-ab59-de5c18447639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:22.126016 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-858d4ff0-5459-47f7-ab59-de5c18447639 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:22.127652 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-858d4ff0-5459-47f7-ab59-de5c18447639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:22.141734 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d0ce1-49b0-443a-aae3-970ffee2a251 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:22.148276 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1154/2316] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:22.156764 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d0ce1-49b0-443a-aae3-970ffee2a251 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:22.169026 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86faea7-4464-4065-9de1-2cab67fb07c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:22.169695 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86faea7-4464-4065-9de1-2cab67fb07c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:22.170396 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86faea7-4464-4065-9de1-2cab67fb07c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:22.170767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86faea7-4464-4065-9de1-2cab67fb07c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:22.176997 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86faea7-4464-4065-9de1-2cab67fb07c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:22.177561 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86faea7-4464-4065-9de1-2cab67fb07c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:22.182585 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86faea7-4464-4065-9de1-2cab67fb07c4 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:22.196125 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1163/2317] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:23:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:23:22.200777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d4ff0-5459-47f7-ab59-de5c18447639 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:22.211162 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d4ff0-5459-47f7-ab59-de5c18447639 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:22.216221 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b2fc5-32d5-4f11-9a18-a5b045ed8d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:22.217090 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b2fc5-32d5-4f11-9a18-a5b045ed8d29 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:22.217369 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b2fc5-32d5-4f11-9a18-a5b045ed8d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:22.217824 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b2fc5-32d5-4f11-9a18-a5b045ed8d29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:22.218520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b2fc5-32d5-4f11-9a18-a5b045ed8d29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:22.220477 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b2fc5-32d5-4f11-9a18-a5b045ed8d29 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:22.262526 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1155/2318] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:23:22 2022] POST /identity/v3/projects => generated 367 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:22.263549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86faea7-4464-4065-9de1-2cab67fb07c4 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:22.276612 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86faea7-4464-4065-9de1-2cab67fb07c4 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:22.333020 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:22.364099 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7886e4-ea7a-4bba-9ffa-25da43cd4355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:22.367534 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7886e4-ea7a-4bba-9ffa-25da43cd4355 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:22.368191 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7886e4-ea7a-4bba-9ffa-25da43cd4355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:22.368967 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7886e4-ea7a-4bba-9ffa-25da43cd4355 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:22.374693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7886e4-ea7a-4bba-9ffa-25da43cd4355 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:22.376296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7886e4-ea7a-4bba-9ffa-25da43cd4355 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:22.407278 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1164/2319] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:23:22 2022] POST /identity/v3/users => generated 284 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:22.411834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b2fc5-32d5-4f11-9a18-a5b045ed8d29 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:22.428761 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b2fc5-32d5-4f11-9a18-a5b045ed8d29 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:22.434299 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0343260-a575-4053-a853-a5c37e7e1400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:22.435086 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0343260-a575-4053-a853-a5c37e7e1400 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:22.435397 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0343260-a575-4053-a853-a5c37e7e1400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:22.435837 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0343260-a575-4053-a853-a5c37e7e1400 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:22.436712 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0343260-a575-4053-a853-a5c37e7e1400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:22.438336 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0343260-a575-4053-a853-a5c37e7e1400 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:22.450999 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1156/2320] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:23:22.455520 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7886e4-ea7a-4bba-9ffa-25da43cd4355 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:22.467574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7886e4-ea7a-4bba-9ffa-25da43cd4355 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:22.473852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3bbdb2-1d44-4351-b4af-a6090a33fcc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:22.474703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3bbdb2-1d44-4351-b4af-a6090a33fcc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:22.475095 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3bbdb2-1d44-4351-b4af-a6090a33fcc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:22.475538 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3bbdb2-1d44-4351-b4af-a6090a33fcc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:22.476459 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3bbdb2-1d44-4351-b4af-a6090a33fcc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:22.478993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3bbdb2-1d44-4351-b4af-a6090a33fcc0 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:22.489560 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1165/2321] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:23:22.493807 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0343260-a575-4053-a853-a5c37e7e1400 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:22.505301 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0343260-a575-4053-a853-a5c37e7e1400 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:22.511029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2232a0b-fef1-4c9b-af7d-df0def21a338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:22.512877 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2232a0b-fef1-4c9b-af7d-df0def21a338 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:22.513756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2232a0b-fef1-4c9b-af7d-df0def21a338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:22.514043 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2232a0b-fef1-4c9b-af7d-df0def21a338 admin admin] PATH_INFO: `/v3/projects/7e1e6566bc3345a69293b67c3edf2677/users/06c1b76b1a5740d1945bbec2cc25a902/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:22.531396 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2232a0b-fef1-4c9b-af7d-df0def21a338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e1e6566bc3345a69293b67c3edf2677, user_id=06c1b76b1a5740d1945bbec2cc25a902, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:22.534044 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2232a0b-fef1-4c9b-af7d-df0def21a338 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:22.537642 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:22.552014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1157/2322] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:23:22 2022] PUT /identity/v3/projects/7e1e6566bc3345a69293b67c3edf2677/users/06c1b76b1a5740d1945bbec2cc25a902/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:23:22.565638 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df51b95-ed26-4256-b810-11be8b5c37a4 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:22.566003 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df51b95-ed26-4256-b810-11be8b5c37a4 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:22.566689 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df51b95-ed26-4256-b810-11be8b5c37a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:22.606586 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-7df51b95-ed26-4256-b810-11be8b5c37a4 None None] MFA Rules not processed for user `06c1b76b1a5740d1945bbec2cc25a902`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:22.608326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7df51b95-ed26-4256-b810-11be8b5c37a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:22.656937 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2232a0b-fef1-4c9b-af7d-df0def21a338 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:22.658750 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e2232a0b-fef1-4c9b-af7d-df0def21a338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:22.709165 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1166/2323] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:22 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:22.725788 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df51b95-ed26-4256-b810-11be8b5c37a4 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:22.728149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7df51b95-ed26-4256-b810-11be8b5c37a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:22.794834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2232a0b-fef1-4c9b-af7d-df0def21a338 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:22.799786 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c5d3c-95c9-47d2-b1d5-de05d10f59b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:22.800821 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c5d3c-95c9-47d2-b1d5-de05d10f59b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:22.801161 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c5d3c-95c9-47d2-b1d5-de05d10f59b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:22.801501 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c5d3c-95c9-47d2-b1d5-de05d10f59b6 admin admin] PATH_INFO: `/v3/users/dea387889c8549df83cb7fc5d2a8f574` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:22.802825 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c5d3c-95c9-47d2-b1d5-de05d10f59b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dea387889c8549df83cb7fc5d2a8f574)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:22.805166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c5d3c-95c9-47d2-b1d5-de05d10f59b6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:22.807137 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df51b95-ed26-4256-b810-11be8b5c37a4 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:22.811827 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7b168-e173-45ee-880d-d813f51c66e8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6180393da0f495db964f0b2588d8b90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:22.812502 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7b168-e173-45ee-880d-d813f51c66e8 service glance] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:22.812786 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7b168-e173-45ee-880d-d813f51c66e8 service glance] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:22.813062 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7b168-e173-45ee-880d-d813f51c66e8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:22.822611 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7b168-e173-45ee-880d-d813f51c66e8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:22.824351 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7b168-e173-45ee-880d-d813f51c66e8 service glance] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:22.834713 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1167/2324] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:22 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:23:22.876830 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-397c5d3c-95c9-47d2-b1d5-de05d10f59b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dea387889c8549df83cb7fc5d2a8f574'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:22.878068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-397c5d3c-95c9-47d2-b1d5-de05d10f59b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dea387889c8549df83cb7fc5d2a8f574'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:22.878757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-397c5d3c-95c9-47d2-b1d5-de05d10f59b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dea387889c8549df83cb7fc5d2a8f574'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:22.888968 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-397c5d3c-95c9-47d2-b1d5-de05d10f59b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dea387889c8549df83cb7fc5d2a8f574'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:22.918849 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-397c5d3c-95c9-47d2-b1d5-de05d10f59b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dea387889c8549df83cb7fc5d2a8f574'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:22.939458 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7b168-e173-45ee-880d-d813f51c66e8 service glance] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:22.943321 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1158/2325] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:23:22 2022] DELETE /identity/v3/users/dea387889c8549df83cb7fc5d2a8f574 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:22.943485 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-afd7b168-e173-45ee-880d-d813f51c66e8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:22.954227 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca45e52-0db9-44bb-b219-e943c29204f7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:22.954602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca45e52-0db9-44bb-b219-e943c29204f7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:22.955102 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca45e52-0db9-44bb-b219-e943c29204f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:22.957873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7b168-e173-45ee-880d-d813f51c66e8 service glance] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:22.989710 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4ca45e52-0db9-44bb-b219-e943c29204f7 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:22.991063 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4ca45e52-0db9-44bb-b219-e943c29204f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:23.015126 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:23.033646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405cb14-7f0a-4e2b-acba-6980c3c73803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:23.034664 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405cb14-7f0a-4e2b-acba-6980c3c73803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:23.035042 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405cb14-7f0a-4e2b-acba-6980c3c73803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:23.035669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405cb14-7f0a-4e2b-acba-6980c3c73803 admin admin] PATH_INFO: `/v3/users/7e838dd1f8464ff490a791441807cd8b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:23.037537 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405cb14-7f0a-4e2b-acba-6980c3c73803 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e838dd1f8464ff490a791441807cd8b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:23.040020 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405cb14-7f0a-4e2b-acba-6980c3c73803 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:23.096596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1159/2326] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:23.126966 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca45e52-0db9-44bb-b219-e943c29204f7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:23.133839 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4ca45e52-0db9-44bb-b219-e943c29204f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:23.155659 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b405cb14-7f0a-4e2b-acba-6980c3c73803 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e838dd1f8464ff490a791441807cd8b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:23.182890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b405cb14-7f0a-4e2b-acba-6980c3c73803 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e838dd1f8464ff490a791441807cd8b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:23.184115 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b405cb14-7f0a-4e2b-acba-6980c3c73803 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e838dd1f8464ff490a791441807cd8b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:23.184855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b405cb14-7f0a-4e2b-acba-6980c3c73803 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e838dd1f8464ff490a791441807cd8b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:23.191675 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:23.216795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b405cb14-7f0a-4e2b-acba-6980c3c73803 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e838dd1f8464ff490a791441807cd8b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:23.246925 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1168/2327] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:23:22 2022] DELETE /identity/v3/users/7e838dd1f8464ff490a791441807cd8b => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:23.251294 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f00f3c-9ed5-40b0-bfe0-aca4eacad309 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:23.251668 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f00f3c-9ed5-40b0-bfe0-aca4eacad309 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:23.252022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f00f3c-9ed5-40b0-bfe0-aca4eacad309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:23.268261 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca45e52-0db9-44bb-b219-e943c29204f7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:23.284553 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-76f00f3c-9ed5-40b0-bfe0-aca4eacad309 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:23.286878 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-76f00f3c-9ed5-40b0-bfe0-aca4eacad309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:23.374524 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1169/2328] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:23.380180 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f00f3c-9ed5-40b0-bfe0-aca4eacad309 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:23.381588 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-76f00f3c-9ed5-40b0-bfe0-aca4eacad309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:23.382112 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371f1c8-804d-4f28-8582-e4a6d06f4853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:23.384030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371f1c8-804d-4f28-8582-e4a6d06f4853 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:23.384565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371f1c8-804d-4f28-8582-e4a6d06f4853 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:23.386378 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371f1c8-804d-4f28-8582-e4a6d06f4853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:23.390968 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8371f1c8-804d-4f28-8582-e4a6d06f4853 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:23.408009 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f00f3c-9ed5-40b0-bfe0-aca4eacad309 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:23.409245 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8371f1c8-804d-4f28-8582-e4a6d06f4853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:23.411562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8371f1c8-804d-4f28-8582-e4a6d06f4853 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:23.414740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313c115-c78b-4f86-aab9-74967c4161b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:23.415536 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313c115-c78b-4f86-aab9-74967c4161b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:23.415814 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313c115-c78b-4f86-aab9-74967c4161b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:23.416113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313c115-c78b-4f86-aab9-74967c4161b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:23.427816 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1160/2329] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:23:23.439767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0313c115-c78b-4f86-aab9-74967c4161b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:23.441809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0313c115-c78b-4f86-aab9-74967c4161b5 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:23.462305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1170/2330] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:23:24.004418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371f1c8-804d-4f28-8582-e4a6d06f4853 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:24.005861 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8371f1c8-804d-4f28-8582-e4a6d06f4853 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:24.019361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371f1c8-804d-4f28-8582-e4a6d06f4853 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:24.024488 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e42136-d8d8-4d44-bcad-9cdf3503e67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:24.025371 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e42136-d8d8-4d44-bcad-9cdf3503e67c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:24.025654 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e42136-d8d8-4d44-bcad-9cdf3503e67c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:24.025929 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e42136-d8d8-4d44-bcad-9cdf3503e67c admin admin] PATH_INFO: `/v3/projects/f171333b618d4e1d80411cfac5544df9` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:24.028919 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e42136-d8d8-4d44-bcad-9cdf3503e67c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f171333b618d4e1d80411cfac5544df9)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:24.030850 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e42136-d8d8-4d44-bcad-9cdf3503e67c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:24.096476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-88e42136-d8d8-4d44-bcad-9cdf3503e67c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f171333b618d4e1d80411cfac5544df9', 'request_id': 'req-88e42136-d8d8-4d44-bcad-9cdf3503e67c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:24.106519 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-88e42136-d8d8-4d44-bcad-9cdf3503e67c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f171333b618d4e1d80411cfac5544df9', 'request_id': 'req-88e42136-d8d8-4d44-bcad-9cdf3503e67c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:24.115018 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-88e42136-d8d8-4d44-bcad-9cdf3503e67c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f171333b618d4e1d80411cfac5544df9', 'request_id': 'req-88e42136-d8d8-4d44-bcad-9cdf3503e67c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:24.129934 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-88e42136-d8d8-4d44-bcad-9cdf3503e67c admin admin] The token cache is being invalidate because project f171333b618d4e1d80411cfac5544df9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:23:24.130227 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-88e42136-d8d8-4d44-bcad-9cdf3503e67c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:24.131147 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-88e42136-d8d8-4d44-bcad-9cdf3503e67c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:24.132933 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1161/2331] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:23:24 2022] DELETE /identity/v3/projects/f171333b618d4e1d80411cfac5544df9 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:24.137709 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313c115-c78b-4f86-aab9-74967c4161b5 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:24.148798 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313c115-c78b-4f86-aab9-74967c4161b5 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:24.210363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:24.222747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a85c68-436f-44a4-87bb-bac7e2d83cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:24.223947 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a85c68-436f-44a4-87bb-bac7e2d83cb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:24.224323 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a85c68-436f-44a4-87bb-bac7e2d83cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:24.224601 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a85c68-436f-44a4-87bb-bac7e2d83cb1 admin admin] PATH_INFO: `/v3/users/47a2da7dc2d849f59c0817f79e339f27` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:24.225804 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a85c68-436f-44a4-87bb-bac7e2d83cb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47a2da7dc2d849f59c0817f79e339f27)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:24.227909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a85c68-436f-44a4-87bb-bac7e2d83cb1 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:24.273231 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e42136-d8d8-4d44-bcad-9cdf3503e67c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:24.274952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-88e42136-d8d8-4d44-bcad-9cdf3503e67c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:24.284470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b7a85c68-436f-44a4-87bb-bac7e2d83cb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47a2da7dc2d849f59c0817f79e339f27'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:24.290526 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e42136-d8d8-4d44-bcad-9cdf3503e67c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:24.296393 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0efb66-bca0-442a-b320-63b9b3eb9531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:24.297280 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0efb66-bca0-442a-b320-63b9b3eb9531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:24.297561 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0efb66-bca0-442a-b320-63b9b3eb9531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:24.297832 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0efb66-bca0-442a-b320-63b9b3eb9531 admin admin] PATH_INFO: `/v3/projects/86d54923b36b4050a1d72199d695eded` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:24.299471 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0efb66-bca0-442a-b320-63b9b3eb9531 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86d54923b36b4050a1d72199d695eded)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:24.301636 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0efb66-bca0-442a-b320-63b9b3eb9531 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:24.305355 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b7a85c68-436f-44a4-87bb-bac7e2d83cb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47a2da7dc2d849f59c0817f79e339f27'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:24.306196 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b7a85c68-436f-44a4-87bb-bac7e2d83cb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47a2da7dc2d849f59c0817f79e339f27'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:24.306953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b7a85c68-436f-44a4-87bb-bac7e2d83cb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47a2da7dc2d849f59c0817f79e339f27'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:24.314418 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b7a85c68-436f-44a4-87bb-bac7e2d83cb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47a2da7dc2d849f59c0817f79e339f27'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:24.329301 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1171/2332] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:23:24 2022] DELETE /identity/v3/users/47a2da7dc2d849f59c0817f79e339f27 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:24.359691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7f0efb66-bca0-442a-b320-63b9b3eb9531 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86d54923b36b4050a1d72199d695eded', 'request_id': 'req-7f0efb66-bca0-442a-b320-63b9b3eb9531'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:24.362927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:24.368403 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7f0efb66-bca0-442a-b320-63b9b3eb9531 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86d54923b36b4050a1d72199d695eded', 'request_id': 'req-7f0efb66-bca0-442a-b320-63b9b3eb9531'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:24.382810 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7f0efb66-bca0-442a-b320-63b9b3eb9531 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86d54923b36b4050a1d72199d695eded', 'request_id': 'req-7f0efb66-bca0-442a-b320-63b9b3eb9531'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:24.399612 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7f0efb66-bca0-442a-b320-63b9b3eb9531 admin admin] The token cache is being invalidate because project 86d54923b36b4050a1d72199d695eded was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:23:24.399905 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7f0efb66-bca0-442a-b320-63b9b3eb9531 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:24.400805 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7f0efb66-bca0-442a-b320-63b9b3eb9531 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:24.403235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1162/2333] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:23:24 2022] DELETE /identity/v3/projects/86d54923b36b4050a1d72199d695eded => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:23:24.408363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df01a9-9fd9-473c-88e5-e5c520c3f9aa None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:24.408738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df01a9-9fd9-473c-88e5-e5c520c3f9aa None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:24.409033 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df01a9-9fd9-473c-88e5-e5c520c3f9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:24.431364 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-34df01a9-9fd9-473c-88e5-e5c520c3f9aa None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:24.432593 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-34df01a9-9fd9-473c-88e5-e5c520c3f9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:24.511719 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1172/2334] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:24.521377 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec1d068-0064-481c-93db-0632389c3ab1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:24.521854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec1d068-0064-481c-93db-0632389c3ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:24.522276 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec1d068-0064-481c-93db-0632389c3ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:24.543277 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-dec1d068-0064-481c-93db-0632389c3ab1 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:24.544528 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dec1d068-0064-481c-93db-0632389c3ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:24.555149 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1163/2335] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:24.559694 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df01a9-9fd9-473c-88e5-e5c520c3f9aa None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:24.570405 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df01a9-9fd9-473c-88e5-e5c520c3f9aa None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:24.576696 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a00d12-775c-46c5-a513-af2be52a313f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:24.577368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a00d12-775c-46c5-a513-af2be52a313f admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:24.577778 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a00d12-775c-46c5-a513-af2be52a313f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:24.578062 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a00d12-775c-46c5-a513-af2be52a313f admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:24.579569 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a00d12-775c-46c5-a513-af2be52a313f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:24.579880 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a00d12-775c-46c5-a513-af2be52a313f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:24.581839 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a00d12-775c-46c5-a513-af2be52a313f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:24.595742 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1173/2336] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:24.600101 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec1d068-0064-481c-93db-0632389c3ab1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:24.610551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec1d068-0064-481c-93db-0632389c3ab1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:24.615754 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b83774a-4fc4-4a30-954c-a3e7aeff3b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:24.616671 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83774a-4fc4-4a30-954c-a3e7aeff3b7f admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:24.617298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83774a-4fc4-4a30-954c-a3e7aeff3b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:24.617775 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83774a-4fc4-4a30-954c-a3e7aeff3b7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:24.618886 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b83774a-4fc4-4a30-954c-a3e7aeff3b7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:24.621965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b83774a-4fc4-4a30-954c-a3e7aeff3b7f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:24.624752 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a00d12-775c-46c5-a513-af2be52a313f admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:24.629063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d5a00d12-775c-46c5-a513-af2be52a313f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:24.655454 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a00d12-775c-46c5-a513-af2be52a313f admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:24.657303 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1164/2337] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:23:24 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:24.662194 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b83774a-4fc4-4a30-954c-a3e7aeff3b7f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:24.724619 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65bcd1-07bc-4483-866f-7992235699b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:24.725527 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65bcd1-07bc-4483-866f-7992235699b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:24.725808 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65bcd1-07bc-4483-866f-7992235699b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:24.726088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65bcd1-07bc-4483-866f-7992235699b1 admin admin] PATH_INFO: `/v3/projects/dcaf9afb5d434e7f99d38774f068cbfd` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:24.727320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c65bcd1-07bc-4483-866f-7992235699b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcaf9afb5d434e7f99d38774f068cbfd)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:24.729233 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c65bcd1-07bc-4483-866f-7992235699b1 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:24.731083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:24.736816 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b83774a-4fc4-4a30-954c-a3e7aeff3b7f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:24.741676 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794bd8ec-7f03-49e8-88b4-8a5fb0a38a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:24.742363 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bd8ec-7f03-49e8-88b4-8a5fb0a38a42 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:24.742649 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bd8ec-7f03-49e8-88b4-8a5fb0a38a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:24.742933 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bd8ec-7f03-49e8-88b4-8a5fb0a38a42 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:24.743664 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794bd8ec-7f03-49e8-88b4-8a5fb0a38a42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:24.746306 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794bd8ec-7f03-49e8-88b4-8a5fb0a38a42 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:24.772879 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1165/2338] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:23:24 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:24.774381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794bd8ec-7f03-49e8-88b4-8a5fb0a38a42 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:24.786203 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794bd8ec-7f03-49e8-88b4-8a5fb0a38a42 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:24.791685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4c65bcd1-07bc-4483-866f-7992235699b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcaf9afb5d434e7f99d38774f068cbfd', 'request_id': 'req-4c65bcd1-07bc-4483-866f-7992235699b1'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:24.805187 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4c65bcd1-07bc-4483-866f-7992235699b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcaf9afb5d434e7f99d38774f068cbfd', 'request_id': 'req-4c65bcd1-07bc-4483-866f-7992235699b1'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:24.813345 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4c65bcd1-07bc-4483-866f-7992235699b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcaf9afb5d434e7f99d38774f068cbfd', 'request_id': 'req-4c65bcd1-07bc-4483-866f-7992235699b1'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:24.829264 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4c65bcd1-07bc-4483-866f-7992235699b1 admin admin] The token cache is being invalidate because project dcaf9afb5d434e7f99d38774f068cbfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:23:24.829649 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4c65bcd1-07bc-4483-866f-7992235699b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:24.830392 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4c65bcd1-07bc-4483-866f-7992235699b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:24.835108 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1174/2339] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:23:24 2022] DELETE /identity/v3/projects/dcaf9afb5d434e7f99d38774f068cbfd => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:24.836197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65bcd1-07bc-4483-866f-7992235699b1 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:24.837733 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4c65bcd1-07bc-4483-866f-7992235699b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:24.858449 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1623d-da9f-4a56-b033-8417b027f222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:24.859139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1623d-da9f-4a56-b033-8417b027f222 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:24.859470 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1623d-da9f-4a56-b033-8417b027f222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:24.859742 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1623d-da9f-4a56-b033-8417b027f222 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:24.860661 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb1623d-da9f-4a56-b033-8417b027f222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:24.862419 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb1623d-da9f-4a56-b033-8417b027f222 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:24.877944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1166/2340] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:23:24.879002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1623d-da9f-4a56-b033-8417b027f222 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:24.880900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8bb1623d-da9f-4a56-b033-8417b027f222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:24.893943 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1623d-da9f-4a56-b033-8417b027f222 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:24.898934 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2cbc1e-e480-4ea6-bc5c-a588b5a7e162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:24.899715 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2cbc1e-e480-4ea6-bc5c-a588b5a7e162 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:24.900010 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2cbc1e-e480-4ea6-bc5c-a588b5a7e162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:24.900292 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2cbc1e-e480-4ea6-bc5c-a588b5a7e162 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:24.901022 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2cbc1e-e480-4ea6-bc5c-a588b5a7e162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:24.903368 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2cbc1e-e480-4ea6-bc5c-a588b5a7e162 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:24.915593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1167/2341] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:23:24.919867 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2cbc1e-e480-4ea6-bc5c-a588b5a7e162 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:24.935117 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2cbc1e-e480-4ea6-bc5c-a588b5a7e162 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:24.940086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65bcd1-07bc-4483-866f-7992235699b1 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:24.944193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc4164c-d15b-43b5-85d6-956cdbc91351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:24.945894 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4164c-d15b-43b5-85d6-956cdbc91351 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:24.946015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637ec7e-2167-44e0-8d18-add559be5542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:24.946110 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4164c-d15b-43b5-85d6-956cdbc91351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:24.946110 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4164c-d15b-43b5-85d6-956cdbc91351 admin admin] PATH_INFO: `/v3/projects/b5977f5c113f44eebed133afa6238c2e/users/86a6598d0d7c41faae54d48ea8a850d7/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:24.946244 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637ec7e-2167-44e0-8d18-add559be5542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:24.946415 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637ec7e-2167-44e0-8d18-add559be5542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:24.946693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637ec7e-2167-44e0-8d18-add559be5542 admin admin] PATH_INFO: `/v3/users/40cf147c05d241cc9341eda110a28822` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:24.948567 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637ec7e-2167-44e0-8d18-add559be5542 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40cf147c05d241cc9341eda110a28822)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:24.950712 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637ec7e-2167-44e0-8d18-add559be5542 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:24.965581 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc4164c-d15b-43b5-85d6-956cdbc91351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5977f5c113f44eebed133afa6238c2e, user_id=86a6598d0d7c41faae54d48ea8a850d7, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:24.969156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc4164c-d15b-43b5-85d6-956cdbc91351 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:24.987400 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1168/2342] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:23:24 2022] PUT /identity/v3/projects/b5977f5c113f44eebed133afa6238c2e/users/86a6598d0d7c41faae54d48ea8a850d7/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:23:24.991433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2dc9f0-8bf3-4474-8ff2-769e10810ee0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:24.991925 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2dc9f0-8bf3-4474-8ff2-769e10810ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:24.992405 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2dc9f0-8bf3-4474-8ff2-769e10810ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:25.000701 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:25.014644 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-fe2dc9f0-8bf3-4474-8ff2-769e10810ee0 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:25.016429 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fe2dc9f0-8bf3-4474-8ff2-769e10810ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:25.016571 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-8637ec7e-2167-44e0-8d18-add559be5542 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40cf147c05d241cc9341eda110a28822'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:25.037362 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-8637ec7e-2167-44e0-8d18-add559be5542 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40cf147c05d241cc9341eda110a28822'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:25.038105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-8637ec7e-2167-44e0-8d18-add559be5542 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40cf147c05d241cc9341eda110a28822'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:25.038762 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-8637ec7e-2167-44e0-8d18-add559be5542 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40cf147c05d241cc9341eda110a28822'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:25.048394 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-8637ec7e-2167-44e0-8d18-add559be5542 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40cf147c05d241cc9341eda110a28822'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:25.069896 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1175/2343] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:23:24 2022] DELETE /identity/v3/users/40cf147c05d241cc9341eda110a28822 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:23:25.103944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1169/2344] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:23:25.114669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637ec7e-2167-44e0-8d18-add559be5542 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:25.116240 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8637ec7e-2167-44e0-8d18-add559be5542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:25.128513 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:25.210846 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637ec7e-2167-44e0-8d18-add559be5542 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:25.218036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682038c-a45d-4d29-a557-3fe80b3bff1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:25.219194 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682038c-a45d-4d29-a557-3fe80b3bff1f service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:25.219902 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682038c-a45d-4d29-a557-3fe80b3bff1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:25.220471 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682038c-a45d-4d29-a557-3fe80b3bff1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:25.298484 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682038c-a45d-4d29-a557-3fe80b3bff1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:25.299956 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682038c-a45d-4d29-a557-3fe80b3bff1f service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:25.309978 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1176/2345] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:23:25.429925 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2dc9f0-8bf3-4474-8ff2-769e10810ee0 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:25.431403 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fe2dc9f0-8bf3-4474-8ff2-769e10810ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:25.444294 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2dc9f0-8bf3-4474-8ff2-769e10810ee0 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:25.450338 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232dd56-1d1c-47cf-8335-cc3f0246a2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:25.451492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232dd56-1d1c-47cf-8335-cc3f0246a2aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:25.452135 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232dd56-1d1c-47cf-8335-cc3f0246a2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:25.452290 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232dd56-1d1c-47cf-8335-cc3f0246a2aa admin admin] PATH_INFO: `/v3/projects/18290df1b4ee407c9cabdbaaedc1f64b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:25.453778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232dd56-1d1c-47cf-8335-cc3f0246a2aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=18290df1b4ee407c9cabdbaaedc1f64b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:25.456151 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232dd56-1d1c-47cf-8335-cc3f0246a2aa admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:25.515537 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6232dd56-1d1c-47cf-8335-cc3f0246a2aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18290df1b4ee407c9cabdbaaedc1f64b', 'request_id': 'req-6232dd56-1d1c-47cf-8335-cc3f0246a2aa'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:25.523751 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6232dd56-1d1c-47cf-8335-cc3f0246a2aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18290df1b4ee407c9cabdbaaedc1f64b', 'request_id': 'req-6232dd56-1d1c-47cf-8335-cc3f0246a2aa'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:25.531023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6232dd56-1d1c-47cf-8335-cc3f0246a2aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18290df1b4ee407c9cabdbaaedc1f64b', 'request_id': 'req-6232dd56-1d1c-47cf-8335-cc3f0246a2aa'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:25.543725 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6232dd56-1d1c-47cf-8335-cc3f0246a2aa admin admin] The token cache is being invalidate because project 18290df1b4ee407c9cabdbaaedc1f64b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:23:25.544167 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6232dd56-1d1c-47cf-8335-cc3f0246a2aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:25.545013 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6232dd56-1d1c-47cf-8335-cc3f0246a2aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:25.551577 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1170/2346] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:23:25 2022] DELETE /identity/v3/projects/18290df1b4ee407c9cabdbaaedc1f64b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:25.553963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88260be1-9ac9-47da-8b46-ec0725e1eb38 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:25.554491 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88260be1-9ac9-47da-8b46-ec0725e1eb38 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:25.554752 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88260be1-9ac9-47da-8b46-ec0725e1eb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:25.578151 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-88260be1-9ac9-47da-8b46-ec0725e1eb38 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:25.579459 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-88260be1-9ac9-47da-8b46-ec0725e1eb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:25.649870 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1177/2347] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:25.658562 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6276a1-f713-4865-ae54-5fdd2ef31422 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:25.658957 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6276a1-f713-4865-ae54-5fdd2ef31422 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:25.659379 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6276a1-f713-4865-ae54-5fdd2ef31422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:25.679207 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1e6276a1-f713-4865-ae54-5fdd2ef31422 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:25.680859 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1e6276a1-f713-4865-ae54-5fdd2ef31422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:25.708210 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1171/2348] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:25.709697 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88260be1-9ac9-47da-8b46-ec0725e1eb38 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:25.727359 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88260be1-9ac9-47da-8b46-ec0725e1eb38 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:25.734751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd60207-daf8-49b6-8f06-8a02deca9b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:25.735542 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd60207-daf8-49b6-8f06-8a02deca9b91 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:25.736189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd60207-daf8-49b6-8f06-8a02deca9b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:25.736546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd60207-daf8-49b6-8f06-8a02deca9b91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:25.737944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd60207-daf8-49b6-8f06-8a02deca9b91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:25.738391 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd60207-daf8-49b6-8f06-8a02deca9b91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:25.740567 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd60207-daf8-49b6-8f06-8a02deca9b91 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:25.757281 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1178/2349] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:25.759702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6276a1-f713-4865-ae54-5fdd2ef31422 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:25.770124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6276a1-f713-4865-ae54-5fdd2ef31422 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:25.775260 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac7fa2-0542-4833-bb2b-f35063b626be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:25.776034 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac7fa2-0542-4833-bb2b-f35063b626be admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:25.776421 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac7fa2-0542-4833-bb2b-f35063b626be admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:25.776808 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac7fa2-0542-4833-bb2b-f35063b626be admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:25.777626 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac7fa2-0542-4833-bb2b-f35063b626be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:25.779704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac7fa2-0542-4833-bb2b-f35063b626be admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:25.812767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1172/2350] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:23:25 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:25.820279 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd60207-daf8-49b6-8f06-8a02deca9b91 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:25.838323 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd60207-daf8-49b6-8f06-8a02deca9b91 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:25.902611 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:25.919365 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c162e5eb-58a5-4d2d-81ff-c60d8a3256a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:25.919734 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162e5eb-58a5-4d2d-81ff-c60d8a3256a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:25.920018 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162e5eb-58a5-4d2d-81ff-c60d8a3256a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:25.920607 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162e5eb-58a5-4d2d-81ff-c60d8a3256a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:25.920987 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c162e5eb-58a5-4d2d-81ff-c60d8a3256a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:25.922902 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c162e5eb-58a5-4d2d-81ff-c60d8a3256a1 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:25.949987 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1179/2351] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:23:25 2022] POST /identity/v3/users => generated 291 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:25.953805 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac7fa2-0542-4833-bb2b-f35063b626be admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:25.968324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac7fa2-0542-4833-bb2b-f35063b626be admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:25.972680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3dcfc-76fa-42ce-98b5-94c01818f727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:25.973797 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3dcfc-76fa-42ce-98b5-94c01818f727 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:25.974126 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3dcfc-76fa-42ce-98b5-94c01818f727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:25.974401 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3dcfc-76fa-42ce-98b5-94c01818f727 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:25.975180 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d3dcfc-76fa-42ce-98b5-94c01818f727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:25.977455 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d3dcfc-76fa-42ce-98b5-94c01818f727 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:25.991872 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1173/2352] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:23:25.997321 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c162e5eb-58a5-4d2d-81ff-c60d8a3256a1 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:26.009037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c162e5eb-58a5-4d2d-81ff-c60d8a3256a1 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:26.014090 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddf48dc-600f-47b8-8f11-103d70e7f4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:26.014838 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf48dc-600f-47b8-8f11-103d70e7f4b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:26.015043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf48dc-600f-47b8-8f11-103d70e7f4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:26.015356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf48dc-600f-47b8-8f11-103d70e7f4b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:26.016099 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddf48dc-600f-47b8-8f11-103d70e7f4b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:26.017882 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddf48dc-600f-47b8-8f11-103d70e7f4b9 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:26.028704 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1180/2353] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:23:26.034165 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3dcfc-76fa-42ce-98b5-94c01818f727 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:26.045628 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3dcfc-76fa-42ce-98b5-94c01818f727 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:26.067655 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2e999-3168-4566-8238-975b701bf50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:26.075743 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2e999-3168-4566-8238-975b701bf50f admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:26.076146 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2e999-3168-4566-8238-975b701bf50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:26.076444 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2e999-3168-4566-8238-975b701bf50f admin admin] PATH_INFO: `/v3/projects/59f9adafa99949b3b5f624c6ae7be57e/users/79357bc4bc0d4b8d9dd5b07a25510f3a/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:26.119421 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2e999-3168-4566-8238-975b701bf50f admin admin] RBAC: Authorizing `identity:create_grant(project_id=59f9adafa99949b3b5f624c6ae7be57e, user_id=79357bc4bc0d4b8d9dd5b07a25510f3a, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:26.123136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2e999-3168-4566-8238-975b701bf50f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:26.155259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1174/2354] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:23:26 2022] PUT /identity/v3/projects/59f9adafa99949b3b5f624c6ae7be57e/users/79357bc4bc0d4b8d9dd5b07a25510f3a/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:26.159890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64114e68-b509-4c53-a712-04bd34d54218 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:26.160185 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64114e68-b509-4c53-a712-04bd34d54218 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:26.160472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64114e68-b509-4c53-a712-04bd34d54218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:26.191039 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-64114e68-b509-4c53-a712-04bd34d54218 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:26.193514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-64114e68-b509-4c53-a712-04bd34d54218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:26.284964 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1181/2355] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:26.320154 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2e999-3168-4566-8238-975b701bf50f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:26.321997 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b7a2e999-3168-4566-8238-975b701bf50f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:26.356451 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:26.397958 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2e999-3168-4566-8238-975b701bf50f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:26.402910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5d122-10ca-4b8d-a770-296b1d8c303a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:26.403977 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5d122-10ca-4b8d-a770-296b1d8c303a service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:26.404268 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5d122-10ca-4b8d-a770-296b1d8c303a service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:26.404546 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5d122-10ca-4b8d-a770-296b1d8c303a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:26.419213 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5d122-10ca-4b8d-a770-296b1d8c303a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:26.423362 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5d122-10ca-4b8d-a770-296b1d8c303a service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:26.438040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1175/2356] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:23:27.623401 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9dd33-1aee-4d19-bb44-958617f74f4b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:27.625266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9dd33-1aee-4d19-bb44-958617f74f4b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:27.625827 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9dd33-1aee-4d19-bb44-958617f74f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:27.678706 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-94c9dd33-1aee-4d19-bb44-958617f74f4b None None] MFA Rules not processed for user `79357bc4bc0d4b8d9dd5b07a25510f3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:27.680022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-94c9dd33-1aee-4d19-bb44-958617f74f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:27.790106 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1182/2357] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:27.803904 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5d122-10ca-4b8d-a770-296b1d8c303a service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:27.805567 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-54c5d122-10ca-4b8d-a770-296b1d8c303a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:27.908195 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5d122-10ca-4b8d-a770-296b1d8c303a service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:27.915275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c834afb-cab7-4673-a671-67136e8bc985 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:27.916109 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c834afb-cab7-4673-a671-67136e8bc985 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:27.916394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c834afb-cab7-4673-a671-67136e8bc985 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:27.916823 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c834afb-cab7-4673-a671-67136e8bc985 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:27.927552 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c834afb-cab7-4673-a671-67136e8bc985 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:27.929238 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c834afb-cab7-4673-a671-67136e8bc985 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:27.941624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1176/2358] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:23:30.465070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf678f-d13a-41c0-8137-673914677178 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:30.465611 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf678f-d13a-41c0-8137-673914677178 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:30.465611 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf678f-d13a-41c0-8137-673914677178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:30.498007 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-b0bf678f-d13a-41c0-8137-673914677178 None None] MFA Rules not processed for user `86a6598d0d7c41faae54d48ea8a850d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:30.499592 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b0bf678f-d13a-41c0-8137-673914677178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:30.555591 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:30.578534 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1183/2359] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:30.584859 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c834afb-cab7-4673-a671-67136e8bc985 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:30.586322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6c834afb-cab7-4673-a671-67136e8bc985 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:30.600786 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c834afb-cab7-4673-a671-67136e8bc985 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:30.606222 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02143f3f-f613-4fb7-9d1a-bc576ac8faef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:30.606895 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02143f3f-f613-4fb7-9d1a-bc576ac8faef admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:30.607179 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02143f3f-f613-4fb7-9d1a-bc576ac8faef admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:30.607488 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02143f3f-f613-4fb7-9d1a-bc576ac8faef admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:30.608203 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02143f3f-f613-4fb7-9d1a-bc576ac8faef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:30.610245 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02143f3f-f613-4fb7-9d1a-bc576ac8faef admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:30.640285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1177/2360] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:23:30 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:30.641588 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf678f-d13a-41c0-8137-673914677178 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:30.654450 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf678f-d13a-41c0-8137-673914677178 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:30.712699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ba3c3-8e2b-4a19-8f9a-a9f2075ac8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:30.713465 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ba3c3-8e2b-4a19-8f9a-a9f2075ac8cc admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:30.714314 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ba3c3-8e2b-4a19-8f9a-a9f2075ac8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:30.714680 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ba3c3-8e2b-4a19-8f9a-a9f2075ac8cc admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:30.715577 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6ba3c3-8e2b-4a19-8f9a-a9f2075ac8cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:30.717589 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6ba3c3-8e2b-4a19-8f9a-a9f2075ac8cc admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:30.742758 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1184/2361] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:23:30 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:30.744181 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02143f3f-f613-4fb7-9d1a-bc576ac8faef admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:30.756447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02143f3f-f613-4fb7-9d1a-bc576ac8faef admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:30.761499 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab35b1d-2292-489c-a688-dd2001fd79dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:30.762173 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab35b1d-2292-489c-a688-dd2001fd79dd admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:30.762452 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab35b1d-2292-489c-a688-dd2001fd79dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:30.762733 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab35b1d-2292-489c-a688-dd2001fd79dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:30.763652 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab35b1d-2292-489c-a688-dd2001fd79dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:30.765243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab35b1d-2292-489c-a688-dd2001fd79dd admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:30.777272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1178/2362] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:23:30.781438 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ba3c3-8e2b-4a19-8f9a-a9f2075ac8cc admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:30.791830 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ba3c3-8e2b-4a19-8f9a-a9f2075ac8cc admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:30.797519 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acdfca0-7f34-4d2e-bf0d-c79494329c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:30.798413 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acdfca0-7f34-4d2e-bf0d-c79494329c76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:30.798698 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acdfca0-7f34-4d2e-bf0d-c79494329c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:30.798974 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acdfca0-7f34-4d2e-bf0d-c79494329c76 admin admin] PATH_INFO: `/v3/projects/02c7033c26d04bdb8e8f091aefb1534b/users/e304d2e6991446ca833973980d124abc/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:30.814594 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acdfca0-7f34-4d2e-bf0d-c79494329c76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02c7033c26d04bdb8e8f091aefb1534b, user_id=e304d2e6991446ca833973980d124abc, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:30.817308 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acdfca0-7f34-4d2e-bf0d-c79494329c76 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:30.835259 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1185/2363] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:23:30 2022] PUT /identity/v3/projects/02c7033c26d04bdb8e8f091aefb1534b/users/e304d2e6991446ca833973980d124abc/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:23:30.838607 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab35b1d-2292-489c-a688-dd2001fd79dd admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:30.855581 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab35b1d-2292-489c-a688-dd2001fd79dd admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:30.923450 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7aa82-52b8-4c27-967d-249d519af30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:30.924147 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7aa82-52b8-4c27-967d-249d519af30d admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:30.924435 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7aa82-52b8-4c27-967d-249d519af30d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:30.924709 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7aa82-52b8-4c27-967d-249d519af30d admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:30.925432 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb7aa82-52b8-4c27-967d-249d519af30d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:30.927049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb7aa82-52b8-4c27-967d-249d519af30d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:30.937310 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1179/2364] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:23:30.941612 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acdfca0-7f34-4d2e-bf0d-c79494329c76 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:30.952508 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acdfca0-7f34-4d2e-bf0d-c79494329c76 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:30.957383 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c9078-fd65-4f18-90a6-c66b069dcb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:30.958270 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c9078-fd65-4f18-90a6-c66b069dcb95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:30.958549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c9078-fd65-4f18-90a6-c66b069dcb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:30.958823 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c9078-fd65-4f18-90a6-c66b069dcb95 admin admin] PATH_INFO: `/v3/projects/02c7033c26d04bdb8e8f091aefb1534b/users/e304d2e6991446ca833973980d124abc/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:30.960858 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c9078-fd65-4f18-90a6-c66b069dcb95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02c7033c26d04bdb8e8f091aefb1534b, user_id=e304d2e6991446ca833973980d124abc, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:30.963608 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c9078-fd65-4f18-90a6-c66b069dcb95 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:30.975573 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1186/2365] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:23:30 2022] PUT /identity/v3/projects/02c7033c26d04bdb8e8f091aefb1534b/users/e304d2e6991446ca833973980d124abc/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:32.692829 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1fec9-984f-4d13-8082-04798fc974b4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:32.693353 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1fec9-984f-4d13-8082-04798fc974b4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:32.693353 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1fec9-984f-4d13-8082-04798fc974b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:32.735536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-92e1fec9-984f-4d13-8082-04798fc974b4 None None] MFA Rules not processed for user `79357bc4bc0d4b8d9dd5b07a25510f3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:32.736714 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-92e1fec9-984f-4d13-8082-04798fc974b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:32.802851 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:32.887000 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1180/2366] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:38.225056 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a3e89-f3a7-4ac5-8792-8ede0f3324dd None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:38.227395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a3e89-f3a7-4ac5-8792-8ede0f3324dd None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:38.227787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a3e89-f3a7-4ac5-8792-8ede0f3324dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:38.274150 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-134a3e89-f3a7-4ac5-8792-8ede0f3324dd None None] MFA Rules not processed for user `e304d2e6991446ca833973980d124abc`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:38.275870 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-134a3e89-f3a7-4ac5-8792-8ede0f3324dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:38.449677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1187/2367] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:38 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:39.071995 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1fec9-984f-4d13-8082-04798fc974b4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:39.074865 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-92e1fec9-984f-4d13-8082-04798fc974b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:39.189159 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1fec9-984f-4d13-8082-04798fc974b4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:39.202333 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5af8f-df74-41fd-b1da-597ce557841f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:39.203466 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5af8f-df74-41fd-b1da-597ce557841f service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:39.204489 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5af8f-df74-41fd-b1da-597ce557841f service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:39.205158 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5af8f-df74-41fd-b1da-597ce557841f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:39.319112 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5af8f-df74-41fd-b1da-597ce557841f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:39.320999 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5af8f-df74-41fd-b1da-597ce557841f service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:39.331759 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1181/2368] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:23:40.857632 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c4cf1-7bc3-4ab0-88d3-d5611d5c44d0 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:40.858884 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c4cf1-7bc3-4ab0-88d3-d5611d5c44d0 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:40.859396 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c4cf1-7bc3-4ab0-88d3-d5611d5c44d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:40.882913 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-bf1c4cf1-7bc3-4ab0-88d3-d5611d5c44d0 None None] MFA Rules not processed for user `86a6598d0d7c41faae54d48ea8a850d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:40.884531 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-bf1c4cf1-7bc3-4ab0-88d3-d5611d5c44d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:40.904705 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1188/2369] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:46.746052 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5af8f-df74-41fd-b1da-597ce557841f service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:46.747662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f5b5af8f-df74-41fd-b1da-597ce557841f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:46.829033 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5af8f-df74-41fd-b1da-597ce557841f service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:46.833820 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cbc881-970c-44f8-990c-3de9d0bae33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:46.834731 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cbc881-970c-44f8-990c-3de9d0bae33c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:46.835018 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cbc881-970c-44f8-990c-3de9d0bae33c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:46.835321 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cbc881-970c-44f8-990c-3de9d0bae33c admin admin] PATH_INFO: `/v3/users/5809fb226baa4664a703ac5c7345cb80` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:46.836557 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cbc881-970c-44f8-990c-3de9d0bae33c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5809fb226baa4664a703ac5c7345cb80)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:46.838653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cbc881-970c-44f8-990c-3de9d0bae33c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:46.864649 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:46.903710 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a0cbc881-970c-44f8-990c-3de9d0bae33c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5809fb226baa4664a703ac5c7345cb80'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:46.904578 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a0cbc881-970c-44f8-990c-3de9d0bae33c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5809fb226baa4664a703ac5c7345cb80'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:46.905191 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a0cbc881-970c-44f8-990c-3de9d0bae33c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5809fb226baa4664a703ac5c7345cb80'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:46.914118 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a0cbc881-970c-44f8-990c-3de9d0bae33c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5809fb226baa4664a703ac5c7345cb80'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:46.928243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a0cbc881-970c-44f8-990c-3de9d0bae33c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5809fb226baa4664a703ac5c7345cb80'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:46.943013 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1182/2370] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:23:46 2022] DELETE /identity/v3/users/5809fb226baa4664a703ac5c7345cb80 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:46.947623 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcb805-308d-46fb-88b4-da612b6f7f8e None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:46.948184 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcb805-308d-46fb-88b4-da612b6f7f8e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:46.948341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcb805-308d-46fb-88b4-da612b6f7f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:46.966478 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-59fcb805-308d-46fb-88b4-da612b6f7f8e None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:46.967721 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-59fcb805-308d-46fb-88b4-da612b6f7f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:46.977102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:47.031423 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1189/2371] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:47.041163 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cbc881-970c-44f8-990c-3de9d0bae33c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:47.043325 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a0cbc881-970c-44f8-990c-3de9d0bae33c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:47.105172 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cbc881-970c-44f8-990c-3de9d0bae33c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:47.108831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1826d6d-761f-4f55-b9f4-3cc509bf9346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:47.109474 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1826d6d-761f-4f55-b9f4-3cc509bf9346 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:47.109756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1826d6d-761f-4f55-b9f4-3cc509bf9346 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:47.110032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1826d6d-761f-4f55-b9f4-3cc509bf9346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:47.123437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1826d6d-761f-4f55-b9f4-3cc509bf9346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:47.124871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1826d6d-761f-4f55-b9f4-3cc509bf9346 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:47.134519 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1183/2372] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:23:47.422139 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fcb805-308d-46fb-88b4-da612b6f7f8e None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:47.423879 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-59fcb805-308d-46fb-88b4-da612b6f7f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:47.436483 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fcb805-308d-46fb-88b4-da612b6f7f8e None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:47.441620 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e14598-e7ee-44a6-9c49-b359d201e95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:47.442488 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e14598-e7ee-44a6-9c49-b359d201e95d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:47.442773 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e14598-e7ee-44a6-9c49-b359d201e95d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:47.443046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e14598-e7ee-44a6-9c49-b359d201e95d admin admin] PATH_INFO: `/v3/projects/8e09c0fb94114711acd6f6b0c2148ec3` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:47.444451 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e14598-e7ee-44a6-9c49-b359d201e95d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e09c0fb94114711acd6f6b0c2148ec3)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:47.446375 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e14598-e7ee-44a6-9c49-b359d201e95d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:47.498160 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-76e14598-e7ee-44a6-9c49-b359d201e95d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e09c0fb94114711acd6f6b0c2148ec3', 'request_id': 'req-76e14598-e7ee-44a6-9c49-b359d201e95d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:47.505475 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-76e14598-e7ee-44a6-9c49-b359d201e95d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e09c0fb94114711acd6f6b0c2148ec3', 'request_id': 'req-76e14598-e7ee-44a6-9c49-b359d201e95d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:47.513729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-76e14598-e7ee-44a6-9c49-b359d201e95d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e09c0fb94114711acd6f6b0c2148ec3', 'request_id': 'req-76e14598-e7ee-44a6-9c49-b359d201e95d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:47.524386 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-76e14598-e7ee-44a6-9c49-b359d201e95d admin admin] The token cache is being invalidate because project 8e09c0fb94114711acd6f6b0c2148ec3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:23:47.524672 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-76e14598-e7ee-44a6-9c49-b359d201e95d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:47.525310 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-76e14598-e7ee-44a6-9c49-b359d201e95d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:47.528159 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1190/2373] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:23:47 2022] DELETE /identity/v3/projects/8e09c0fb94114711acd6f6b0c2148ec3 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:47.532109 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1826d6d-761f-4f55-b9f4-3cc509bf9346 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:47.542275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1826d6d-761f-4f55-b9f4-3cc509bf9346 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:47.600182 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af24c3e-79f6-4820-85c9-719965e1be23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:47.601057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af24c3e-79f6-4820-85c9-719965e1be23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:47.601341 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af24c3e-79f6-4820-85c9-719965e1be23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:47.601618 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af24c3e-79f6-4820-85c9-719965e1be23 admin admin] PATH_INFO: `/v3/users/ebaa488d45374133aee1bad1d6b091e7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:47.602830 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af24c3e-79f6-4820-85c9-719965e1be23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebaa488d45374133aee1bad1d6b091e7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:47.605265 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af24c3e-79f6-4820-85c9-719965e1be23 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:47.644692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:47.655167 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1af24c3e-79f6-4820-85c9-719965e1be23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebaa488d45374133aee1bad1d6b091e7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:47.656060 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1af24c3e-79f6-4820-85c9-719965e1be23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebaa488d45374133aee1bad1d6b091e7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:47.656688 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1af24c3e-79f6-4820-85c9-719965e1be23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebaa488d45374133aee1bad1d6b091e7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:47.663870 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1af24c3e-79f6-4820-85c9-719965e1be23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebaa488d45374133aee1bad1d6b091e7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:47.675804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1af24c3e-79f6-4820-85c9-719965e1be23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebaa488d45374133aee1bad1d6b091e7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:47.692601 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1184/2374] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:23:47 2022] DELETE /identity/v3/users/ebaa488d45374133aee1bad1d6b091e7 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:23:48.050523 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e14598-e7ee-44a6-9c49-b359d201e95d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:48.052055 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-76e14598-e7ee-44a6-9c49-b359d201e95d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:48.071329 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e14598-e7ee-44a6-9c49-b359d201e95d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:48.164685 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabcd381-54ae-403c-9e3c-9630a25f80ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:48.169812 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcd381-54ae-403c-9e3c-9630a25f80ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:48.170282 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcd381-54ae-403c-9e3c-9630a25f80ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:48.170711 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcd381-54ae-403c-9e3c-9630a25f80ff admin admin] PATH_INFO: `/v3/projects/6414ec0a008f473881894cdeaa7aabe8` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:48.172619 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabcd381-54ae-403c-9e3c-9630a25f80ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=6414ec0a008f473881894cdeaa7aabe8)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:48.175133 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabcd381-54ae-403c-9e3c-9630a25f80ff admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:48.222330 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:48.235063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-aabcd381-54ae-403c-9e3c-9630a25f80ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6414ec0a008f473881894cdeaa7aabe8', 'request_id': 'req-aabcd381-54ae-403c-9e3c-9630a25f80ff'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:48.249143 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-aabcd381-54ae-403c-9e3c-9630a25f80ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6414ec0a008f473881894cdeaa7aabe8', 'request_id': 'req-aabcd381-54ae-403c-9e3c-9630a25f80ff'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:48.261230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-aabcd381-54ae-403c-9e3c-9630a25f80ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6414ec0a008f473881894cdeaa7aabe8', 'request_id': 'req-aabcd381-54ae-403c-9e3c-9630a25f80ff'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:48.277064 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-aabcd381-54ae-403c-9e3c-9630a25f80ff admin admin] The token cache is being invalidate because project 6414ec0a008f473881894cdeaa7aabe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:23:48.277408 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-aabcd381-54ae-403c-9e3c-9630a25f80ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:48.278081 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-aabcd381-54ae-403c-9e3c-9630a25f80ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:48.284395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1191/2375] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:23:48 2022] DELETE /identity/v3/projects/6414ec0a008f473881894cdeaa7aabe8 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:48.285808 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f5cc5-8ade-4b9d-9791-3d144045b51d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:48.286353 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f5cc5-8ade-4b9d-9791-3d144045b51d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:48.286642 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f5cc5-8ade-4b9d-9791-3d144045b51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:48.378074 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-941f5cc5-8ade-4b9d-9791-3d144045b51d None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:48.380833 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-941f5cc5-8ade-4b9d-9791-3d144045b51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:48.519059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1185/2376] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:23:48.528200 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ac8b1-bf27-45a2-aa37-fb8103fd02e4 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:48.528722 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ac8b1-bf27-45a2-aa37-fb8103fd02e4 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:48.529204 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ac8b1-bf27-45a2-aa37-fb8103fd02e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:48.575834 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-d92ac8b1-bf27-45a2-aa37-fb8103fd02e4 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:48.577193 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d92ac8b1-bf27-45a2-aa37-fb8103fd02e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:48.598469 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1192/2377] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:48.600889 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f5cc5-8ade-4b9d-9791-3d144045b51d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:48.634496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f5cc5-8ade-4b9d-9791-3d144045b51d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:48.653953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9fb82-24c8-452c-9ebe-040703bd038d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:48.654665 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9fb82-24c8-452c-9ebe-040703bd038d admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:48.654955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9fb82-24c8-452c-9ebe-040703bd038d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:48.655265 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9fb82-24c8-452c-9ebe-040703bd038d admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:48.656032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f9fb82-24c8-452c-9ebe-040703bd038d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:48.656348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f9fb82-24c8-452c-9ebe-040703bd038d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:48.658059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f9fb82-24c8-452c-9ebe-040703bd038d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:48.683627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1186/2378] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:23:48.684932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ac8b1-bf27-45a2-aa37-fb8103fd02e4 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:48.696948 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ac8b1-bf27-45a2-aa37-fb8103fd02e4 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:48.732066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36c2de-d776-4ab8-97c8-c922c126bde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:48.732768 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36c2de-d776-4ab8-97c8-c922c126bde3 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:48.733050 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36c2de-d776-4ab8-97c8-c922c126bde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:48.733329 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36c2de-d776-4ab8-97c8-c922c126bde3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:48.734034 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36c2de-d776-4ab8-97c8-c922c126bde3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:48.735988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36c2de-d776-4ab8-97c8-c922c126bde3 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:48.787847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1193/2379] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:23:48 2022] POST /identity/v3/projects => generated 367 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:48.792118 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9fb82-24c8-452c-9ebe-040703bd038d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:48.806620 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9fb82-24c8-452c-9ebe-040703bd038d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:48.920791 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5badf8b-01ae-4a82-953c-749bffc081d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:48.921721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5badf8b-01ae-4a82-953c-749bffc081d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:48.922293 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5badf8b-01ae-4a82-953c-749bffc081d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:48.922553 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5badf8b-01ae-4a82-953c-749bffc081d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:48.924208 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5badf8b-01ae-4a82-953c-749bffc081d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:48.926300 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5badf8b-01ae-4a82-953c-749bffc081d7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:48.955963 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1187/2380] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:23:48 2022] POST /identity/v3/users => generated 284 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:48.956563 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36c2de-d776-4ab8-97c8-c922c126bde3 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:48.969492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36c2de-d776-4ab8-97c8-c922c126bde3 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:48.977680 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7dc22-ca83-4351-a9d6-81f954e78df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:48.980438 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7dc22-ca83-4351-a9d6-81f954e78df3 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:48.981307 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7dc22-ca83-4351-a9d6-81f954e78df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:48.982167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7dc22-ca83-4351-a9d6-81f954e78df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:48.982650 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7dc22-ca83-4351-a9d6-81f954e78df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:48.984712 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7dc22-ca83-4351-a9d6-81f954e78df3 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:49.023708 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1194/2381] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:23:49.029498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5badf8b-01ae-4a82-953c-749bffc081d7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:49.045915 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5badf8b-01ae-4a82-953c-749bffc081d7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:49.051342 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3abc1-7945-4afc-bdd0-b17eed224bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:49.052023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3abc1-7945-4afc-bdd0-b17eed224bdc admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:49.052308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3abc1-7945-4afc-bdd0-b17eed224bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:49.052583 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3abc1-7945-4afc-bdd0-b17eed224bdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:49.053311 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3abc1-7945-4afc-bdd0-b17eed224bdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:49.054937 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3abc1-7945-4afc-bdd0-b17eed224bdc admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:49.069168 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1188/2382] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:23:49.073636 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7dc22-ca83-4351-a9d6-81f954e78df3 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:49.092706 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7dc22-ca83-4351-a9d6-81f954e78df3 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:49.107161 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857588a6-2f5d-4cb9-b90d-b7982a1e54ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:49.108747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857588a6-2f5d-4cb9-b90d-b7982a1e54ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:49.109248 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857588a6-2f5d-4cb9-b90d-b7982a1e54ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:49.109779 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857588a6-2f5d-4cb9-b90d-b7982a1e54ff admin admin] PATH_INFO: `/v3/projects/a6f09f6f44ae4d1f88c62040a2aa4b88/users/c6348145687941c2a4f6bfd0f149677f/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:49.132272 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857588a6-2f5d-4cb9-b90d-b7982a1e54ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6f09f6f44ae4d1f88c62040a2aa4b88, user_id=c6348145687941c2a4f6bfd0f149677f, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:49.135558 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857588a6-2f5d-4cb9-b90d-b7982a1e54ff admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:49.194739 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bf376-4ab9-43c0-84b7-82cd426c8586 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:49.195030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bf376-4ab9-43c0-84b7-82cd426c8586 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:49.195377 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bf376-4ab9-43c0-84b7-82cd426c8586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:49.199312 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1195/2383] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:23:49 2022] PUT /identity/v3/projects/a6f09f6f44ae4d1f88c62040a2aa4b88/users/c6348145687941c2a4f6bfd0f149677f/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:23:49.222048 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:49.238297 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d84bf376-4ab9-43c0-84b7-82cd426c8586 None None] MFA Rules not processed for user `c6348145687941c2a4f6bfd0f149677f`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:49.239503 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d84bf376-4ab9-43c0-84b7-82cd426c8586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:49.332705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1189/2384] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:49 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:49.337033 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857588a6-2f5d-4cb9-b90d-b7982a1e54ff admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:49.417737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857588a6-2f5d-4cb9-b90d-b7982a1e54ff admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:49.422408 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6d825-3c21-4628-a498-db2fd3711892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:49.423074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6d825-3c21-4628-a498-db2fd3711892 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:49.423384 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6d825-3c21-4628-a498-db2fd3711892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:49.423661 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6d825-3c21-4628-a498-db2fd3711892 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:49.424351 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6d825-3c21-4628-a498-db2fd3711892 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:49.426242 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6d825-3c21-4628-a498-db2fd3711892 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:49.454140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1196/2385] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:23:49 2022] POST /identity/v3/projects => generated 365 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:23:49.458516 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84bf376-4ab9-43c0-84b7-82cd426c8586 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:49.470726 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84bf376-4ab9-43c0-84b7-82cd426c8586 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:49.532847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153766f-30f1-499c-9609-8d364ff527f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:49.533521 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153766f-30f1-499c-9609-8d364ff527f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:49.533802 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153766f-30f1-499c-9609-8d364ff527f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:49.534170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153766f-30f1-499c-9609-8d364ff527f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:49.535502 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a153766f-30f1-499c-9609-8d364ff527f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:49.538831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a153766f-30f1-499c-9609-8d364ff527f0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:49.566642 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1190/2386] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:23:49 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:49.568189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6d825-3c21-4628-a498-db2fd3711892 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:49.579775 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6d825-3c21-4628-a498-db2fd3711892 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:49.584642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d711e2-fbef-4585-88ea-3dc491fc5e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:49.585303 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d711e2-fbef-4585-88ea-3dc491fc5e4d admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:49.585578 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d711e2-fbef-4585-88ea-3dc491fc5e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:49.585851 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d711e2-fbef-4585-88ea-3dc491fc5e4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:49.586570 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d711e2-fbef-4585-88ea-3dc491fc5e4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:49.588974 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d711e2-fbef-4585-88ea-3dc491fc5e4d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:49.601484 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1197/2387] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:23:49.605600 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153766f-30f1-499c-9609-8d364ff527f0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:49.615628 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153766f-30f1-499c-9609-8d364ff527f0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:49.621643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261680e7-962c-4501-95ec-763720ff13c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:49.622533 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261680e7-962c-4501-95ec-763720ff13c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:49.622811 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261680e7-962c-4501-95ec-763720ff13c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:49.623087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261680e7-962c-4501-95ec-763720ff13c3 admin admin] PATH_INFO: `/v3/projects/dfbf6811f3f74c97927f9cf12eb0bc4f/users/1c7207cc0df848b5a33f9c812fb04cf4/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:49.639555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261680e7-962c-4501-95ec-763720ff13c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfbf6811f3f74c97927f9cf12eb0bc4f, user_id=1c7207cc0df848b5a33f9c812fb04cf4, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:49.642149 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261680e7-962c-4501-95ec-763720ff13c3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:49.655734 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1191/2388] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:23:49 2022] PUT /identity/v3/projects/dfbf6811f3f74c97927f9cf12eb0bc4f/users/1c7207cc0df848b5a33f9c812fb04cf4/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:49.659996 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d711e2-fbef-4585-88ea-3dc491fc5e4d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:49.670395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d711e2-fbef-4585-88ea-3dc491fc5e4d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:49.710056 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:49.732022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e01eb-b916-438b-87f9-6c97a166b0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:49.732698 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e01eb-b916-438b-87f9-6c97a166b0c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:49.732983 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e01eb-b916-438b-87f9-6c97a166b0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:49.733259 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e01eb-b916-438b-87f9-6c97a166b0c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:49.733995 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7e01eb-b916-438b-87f9-6c97a166b0c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:49.735658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7e01eb-b916-438b-87f9-6c97a166b0c3 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:49.750095 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1198/2389] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:23:49.751515 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261680e7-962c-4501-95ec-763720ff13c3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:49.762588 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261680e7-962c-4501-95ec-763720ff13c3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:49.767648 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe08a0-d0a4-4dcd-afad-d0e9c2fbcaf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:49.768663 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe08a0-d0a4-4dcd-afad-d0e9c2fbcaf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:49.769047 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe08a0-d0a4-4dcd-afad-d0e9c2fbcaf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:49.769468 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe08a0-d0a4-4dcd-afad-d0e9c2fbcaf8 admin admin] PATH_INFO: `/v3/projects/dfbf6811f3f74c97927f9cf12eb0bc4f/users/1c7207cc0df848b5a33f9c812fb04cf4/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:49.771594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe08a0-d0a4-4dcd-afad-d0e9c2fbcaf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfbf6811f3f74c97927f9cf12eb0bc4f, user_id=1c7207cc0df848b5a33f9c812fb04cf4, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:49.774400 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe08a0-d0a4-4dcd-afad-d0e9c2fbcaf8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:49.786097 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1192/2390] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:23:49 2022] PUT /identity/v3/projects/dfbf6811f3f74c97927f9cf12eb0bc4f/users/1c7207cc0df848b5a33f9c812fb04cf4/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:49.797911 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd25177-0aa5-4ec5-83d3-4277070e62b7 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:49.798464 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd25177-0aa5-4ec5-83d3-4277070e62b7 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:49.798891 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd25177-0aa5-4ec5-83d3-4277070e62b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:49.831842 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-2fd25177-0aa5-4ec5-83d3-4277070e62b7 None None] MFA Rules not processed for user `1c7207cc0df848b5a33f9c812fb04cf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:49.833500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2fd25177-0aa5-4ec5-83d3-4277070e62b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:49.918794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1199/2391] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:49 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:49.930436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe08a0-d0a4-4dcd-afad-d0e9c2fbcaf8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:49.933105 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cebe08a0-d0a4-4dcd-afad-d0e9c2fbcaf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:49.992112 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe08a0-d0a4-4dcd-afad-d0e9c2fbcaf8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:49.996299 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca777aee-7627-4467-b230-aac6134d1bbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:49.999005 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca777aee-7627-4467-b230-aac6134d1bbd service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:49.999317 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca777aee-7627-4467-b230-aac6134d1bbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:49.999596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca777aee-7627-4467-b230-aac6134d1bbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:50.006974 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:50.009623 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca777aee-7627-4467-b230-aac6134d1bbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:50.013276 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca777aee-7627-4467-b230-aac6134d1bbd service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:50.023243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1193/2392] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:49 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:23:50.064499 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd25177-0aa5-4ec5-83d3-4277070e62b7 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:50.065895 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2fd25177-0aa5-4ec5-83d3-4277070e62b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:50.078429 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd25177-0aa5-4ec5-83d3-4277070e62b7 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:50.082084 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb11d4-3b3d-4136-9e05-ce38afc06562 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:50.082723 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb11d4-3b3d-4136-9e05-ce38afc06562 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:50.083008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb11d4-3b3d-4136-9e05-ce38afc06562 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:50.083324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb11d4-3b3d-4136-9e05-ce38afc06562 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:50.141094 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb11d4-3b3d-4136-9e05-ce38afc06562 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:50.142789 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb11d4-3b3d-4136-9e05-ce38afc06562 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:50.156987 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1200/2393] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:50 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:23:53.706137 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510c74a-1d2d-4274-b0c8-8c81d20fd0f7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:53.706686 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510c74a-1d2d-4274-b0c8-8c81d20fd0f7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:53.706799 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510c74a-1d2d-4274-b0c8-8c81d20fd0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:53.725348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4510c74a-1d2d-4274-b0c8-8c81d20fd0f7 None None] MFA Rules not processed for user `becdcd1df4a641e6a5bda05220f7aa07`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:53.726548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4510c74a-1d2d-4274-b0c8-8c81d20fd0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:53.759949 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb11d4-3b3d-4136-9e05-ce38afc06562 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:53.790835 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1194/2394] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:53.835587 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb11d4-3b3d-4136-9e05-ce38afc06562 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:53.840562 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8e793-258a-4b7b-a1d8-8436587ba54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:53.841438 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8e793-258a-4b7b-a1d8-8436587ba54b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:53.841720 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8e793-258a-4b7b-a1d8-8436587ba54b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:53.842361 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8e793-258a-4b7b-a1d8-8436587ba54b admin admin] PATH_INFO: `/v3/users/c6348145687941c2a4f6bfd0f149677f` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:53.843622 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8e793-258a-4b7b-a1d8-8436587ba54b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6348145687941c2a4f6bfd0f149677f)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:53.845636 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8e793-258a-4b7b-a1d8-8436587ba54b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:53.847397 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:53.909064 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b0d8e793-258a-4b7b-a1d8-8436587ba54b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6348145687941c2a4f6bfd0f149677f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:53.929016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b0d8e793-258a-4b7b-a1d8-8436587ba54b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6348145687941c2a4f6bfd0f149677f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:53.929783 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b0d8e793-258a-4b7b-a1d8-8436587ba54b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6348145687941c2a4f6bfd0f149677f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:53.930400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b0d8e793-258a-4b7b-a1d8-8436587ba54b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6348145687941c2a4f6bfd0f149677f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:53.938225 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b0d8e793-258a-4b7b-a1d8-8436587ba54b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6348145687941c2a4f6bfd0f149677f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:53.957691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1201/2395] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:23:53 2022] DELETE /identity/v3/users/c6348145687941c2a4f6bfd0f149677f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:23:53.965260 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534864d6-a709-42e5-ba63-aa84ad916bec None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:53.966271 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534864d6-a709-42e5-ba63-aa84ad916bec None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:53.966576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534864d6-a709-42e5-ba63-aa84ad916bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:53.988984 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-534864d6-a709-42e5-ba63-aa84ad916bec None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:53.990784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-534864d6-a709-42e5-ba63-aa84ad916bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:54.061375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1195/2396] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:54.074729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8e793-258a-4b7b-a1d8-8436587ba54b admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:54.076471 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b0d8e793-258a-4b7b-a1d8-8436587ba54b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:54.164343 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8e793-258a-4b7b-a1d8-8436587ba54b admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:54.168657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f6db9-c26e-499d-841b-9abf1daf486d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:54.169329 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f6db9-c26e-499d-841b-9abf1daf486d service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:54.169607 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f6db9-c26e-499d-841b-9abf1daf486d service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:54.169886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f6db9-c26e-499d-841b-9abf1daf486d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:54.184890 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f6db9-c26e-499d-841b-9abf1daf486d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:54.186457 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f6db9-c26e-499d-841b-9abf1daf486d service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:54.197335 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1202/2397] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:23:54.576751 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534864d6-a709-42e5-ba63-aa84ad916bec None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:54.579475 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-534864d6-a709-42e5-ba63-aa84ad916bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:54.612306 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534864d6-a709-42e5-ba63-aa84ad916bec None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:54.627568 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b725ed2-8b59-42fd-97b2-c83252f2bdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:54.627568 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b725ed2-8b59-42fd-97b2-c83252f2bdd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:54.627568 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b725ed2-8b59-42fd-97b2-c83252f2bdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:54.627568 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b725ed2-8b59-42fd-97b2-c83252f2bdd7 admin admin] PATH_INFO: `/v3/projects/a6f09f6f44ae4d1f88c62040a2aa4b88` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:54.628244 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b725ed2-8b59-42fd-97b2-c83252f2bdd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6f09f6f44ae4d1f88c62040a2aa4b88)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:54.630820 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b725ed2-8b59-42fd-97b2-c83252f2bdd7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:54.703030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3b725ed2-8b59-42fd-97b2-c83252f2bdd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6f09f6f44ae4d1f88c62040a2aa4b88', 'request_id': 'req-3b725ed2-8b59-42fd-97b2-c83252f2bdd7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:54.715109 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3b725ed2-8b59-42fd-97b2-c83252f2bdd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6f09f6f44ae4d1f88c62040a2aa4b88', 'request_id': 'req-3b725ed2-8b59-42fd-97b2-c83252f2bdd7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:54.724170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3b725ed2-8b59-42fd-97b2-c83252f2bdd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6f09f6f44ae4d1f88c62040a2aa4b88', 'request_id': 'req-3b725ed2-8b59-42fd-97b2-c83252f2bdd7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:54.754062 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3b725ed2-8b59-42fd-97b2-c83252f2bdd7 admin admin] The token cache is being invalidate because project a6f09f6f44ae4d1f88c62040a2aa4b88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:23:54.754363 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3b725ed2-8b59-42fd-97b2-c83252f2bdd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:54.755629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3b725ed2-8b59-42fd-97b2-c83252f2bdd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:54.758601 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1196/2398] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:23:54 2022] DELETE /identity/v3/projects/a6f09f6f44ae4d1f88c62040a2aa4b88 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:54.763371 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f6db9-c26e-499d-841b-9abf1daf486d service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:54.784504 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f6db9-c26e-499d-841b-9abf1daf486d service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:54.867169 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2cd80-7798-4aa3-ad1f-4729a0cbbbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:54.868718 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2cd80-7798-4aa3-ad1f-4729a0cbbbd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:54.869682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2cd80-7798-4aa3-ad1f-4729a0cbbbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:54.870126 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2cd80-7798-4aa3-ad1f-4729a0cbbbd4 admin admin] PATH_INFO: `/v3/users/1c7207cc0df848b5a33f9c812fb04cf4` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:54.874579 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2cd80-7798-4aa3-ad1f-4729a0cbbbd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c7207cc0df848b5a33f9c812fb04cf4)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:54.877017 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2cd80-7798-4aa3-ad1f-4729a0cbbbd4 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:54.932842 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:54.983609 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-62f2cd80-7798-4aa3-ad1f-4729a0cbbbd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c7207cc0df848b5a33f9c812fb04cf4'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:55.005165 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-62f2cd80-7798-4aa3-ad1f-4729a0cbbbd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c7207cc0df848b5a33f9c812fb04cf4'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:55.006163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-62f2cd80-7798-4aa3-ad1f-4729a0cbbbd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c7207cc0df848b5a33f9c812fb04cf4'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:55.007050 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-62f2cd80-7798-4aa3-ad1f-4729a0cbbbd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c7207cc0df848b5a33f9c812fb04cf4'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:55.016968 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-62f2cd80-7798-4aa3-ad1f-4729a0cbbbd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c7207cc0df848b5a33f9c812fb04cf4'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:55.031213 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1203/2399] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:23:54 2022] DELETE /identity/v3/users/1c7207cc0df848b5a33f9c812fb04cf4 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:55.329151 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b725ed2-8b59-42fd-97b2-c83252f2bdd7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:55.331201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3b725ed2-8b59-42fd-97b2-c83252f2bdd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:55.363762 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b725ed2-8b59-42fd-97b2-c83252f2bdd7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:55.389361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:55.433404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a5dc9-c337-420f-b58b-f2b02cc2c603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:55.434004 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a5dc9-c337-420f-b58b-f2b02cc2c603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:55.434294 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a5dc9-c337-420f-b58b-f2b02cc2c603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:55.434572 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a5dc9-c337-420f-b58b-f2b02cc2c603 admin admin] PATH_INFO: `/v3/projects/dfbf6811f3f74c97927f9cf12eb0bc4f` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:55.435902 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06a5dc9-c337-420f-b58b-f2b02cc2c603 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfbf6811f3f74c97927f9cf12eb0bc4f)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:55.438178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06a5dc9-c337-420f-b58b-f2b02cc2c603 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:55.496308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d06a5dc9-c337-420f-b58b-f2b02cc2c603 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfbf6811f3f74c97927f9cf12eb0bc4f', 'request_id': 'req-d06a5dc9-c337-420f-b58b-f2b02cc2c603'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:55.504616 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d06a5dc9-c337-420f-b58b-f2b02cc2c603 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfbf6811f3f74c97927f9cf12eb0bc4f', 'request_id': 'req-d06a5dc9-c337-420f-b58b-f2b02cc2c603'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:55.513604 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d06a5dc9-c337-420f-b58b-f2b02cc2c603 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfbf6811f3f74c97927f9cf12eb0bc4f', 'request_id': 'req-d06a5dc9-c337-420f-b58b-f2b02cc2c603'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:55.531185 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d06a5dc9-c337-420f-b58b-f2b02cc2c603 admin admin] The token cache is being invalidate because project dfbf6811f3f74c97927f9cf12eb0bc4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:23:55.531943 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d06a5dc9-c337-420f-b58b-f2b02cc2c603 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:55.532619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d06a5dc9-c337-420f-b58b-f2b02cc2c603 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:55.535127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1197/2400] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:23:55 2022] DELETE /identity/v3/projects/dfbf6811f3f74c97927f9cf12eb0bc4f => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:55.539467 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cdacfa-ae71-4691-9496-aa2c8e094a6b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:55.539467 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cdacfa-ae71-4691-9496-aa2c8e094a6b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:55.539720 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cdacfa-ae71-4691-9496-aa2c8e094a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:55.562642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-05cdacfa-ae71-4691-9496-aa2c8e094a6b None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:55.564227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-05cdacfa-ae71-4691-9496-aa2c8e094a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:55.662187 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1204/2401] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:55.670768 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2df53-8c50-4e96-9020-ba991f398fc9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:55.671120 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2df53-8c50-4e96-9020-ba991f398fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:55.672507 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2df53-8c50-4e96-9020-ba991f398fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:55.700512 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ebc2df53-8c50-4e96-9020-ba991f398fc9 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:55.702168 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ebc2df53-8c50-4e96-9020-ba991f398fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:55.717420 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1198/2402] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:55.718086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cdacfa-ae71-4691-9496-aa2c8e094a6b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:55.733555 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cdacfa-ae71-4691-9496-aa2c8e094a6b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:55.753538 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c0202-0d2b-43f3-a4a7-95ef1c905432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:55.756086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c0202-0d2b-43f3-a4a7-95ef1c905432 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:55.756086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c0202-0d2b-43f3-a4a7-95ef1c905432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:55.756086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c0202-0d2b-43f3-a4a7-95ef1c905432 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:55.756086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c0202-0d2b-43f3-a4a7-95ef1c905432 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:55.756591 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c0202-0d2b-43f3-a4a7-95ef1c905432 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:55.758157 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c0202-0d2b-43f3-a4a7-95ef1c905432 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:55.777344 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1205/2403] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:23:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:55.781387 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2df53-8c50-4e96-9020-ba991f398fc9 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:55.792240 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2df53-8c50-4e96-9020-ba991f398fc9 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:55.798689 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6858dab-014c-47b6-8daa-c3d431770399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:55.799500 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6858dab-014c-47b6-8daa-c3d431770399 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:55.799789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6858dab-014c-47b6-8daa-c3d431770399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:55.800484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6858dab-014c-47b6-8daa-c3d431770399 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:55.801496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6858dab-014c-47b6-8daa-c3d431770399 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:55.804361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6858dab-014c-47b6-8daa-c3d431770399 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:55.855604 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1199/2404] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:23:55 2022] POST /identity/v3/projects => generated 383 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:55.860043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c0202-0d2b-43f3-a4a7-95ef1c905432 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:55.872403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c0202-0d2b-43f3-a4a7-95ef1c905432 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:55.945810 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2647f100-5183-4ecd-a665-6cfd3034202c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:55.946665 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647f100-5183-4ecd-a665-6cfd3034202c admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:55.947016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647f100-5183-4ecd-a665-6cfd3034202c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:55.947520 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647f100-5183-4ecd-a665-6cfd3034202c admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:55.948652 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2647f100-5183-4ecd-a665-6cfd3034202c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:55.950891 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2647f100-5183-4ecd-a665-6cfd3034202c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:55.978110 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1206/2405] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:23:55 2022] POST /identity/v3/users => generated 292 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:55.982578 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6858dab-014c-47b6-8daa-c3d431770399 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:55.996712 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6858dab-014c-47b6-8daa-c3d431770399 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:56.011500 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b0175-9fd1-4340-b118-403cf2b49f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:56.012599 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b0175-9fd1-4340-b118-403cf2b49f9f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:56.012946 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b0175-9fd1-4340-b118-403cf2b49f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:56.013286 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b0175-9fd1-4340-b118-403cf2b49f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:56.014146 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b0175-9fd1-4340-b118-403cf2b49f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:56.015995 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b0175-9fd1-4340-b118-403cf2b49f9f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:56.027714 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1200/2406] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:23:56.031621 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2647f100-5183-4ecd-a665-6cfd3034202c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:56.042652 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2647f100-5183-4ecd-a665-6cfd3034202c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:56.047590 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09149460-a1e5-4abf-9d2a-489c969b9cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:56.048332 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09149460-a1e5-4abf-9d2a-489c969b9cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:56.048686 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09149460-a1e5-4abf-9d2a-489c969b9cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:56.048969 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09149460-a1e5-4abf-9d2a-489c969b9cf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:56.049894 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09149460-a1e5-4abf-9d2a-489c969b9cf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:56.051543 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09149460-a1e5-4abf-9d2a-489c969b9cf9 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:56.068027 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1207/2407] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:23:56.069190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b0175-9fd1-4340-b118-403cf2b49f9f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:56.079756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b0175-9fd1-4340-b118-403cf2b49f9f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:56.089780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7cad2-4cea-4d41-91ff-c57291dd54b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:56.091008 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7cad2-4cea-4d41-91ff-c57291dd54b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:56.091318 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7cad2-4cea-4d41-91ff-c57291dd54b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:56.091588 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7cad2-4cea-4d41-91ff-c57291dd54b4 admin admin] PATH_INFO: `/v3/projects/0ffaba31c9744c46ac5cfcb38b9e934f/users/5afa7c67bc80425a90dddac0e33f8424/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:56.107725 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f7cad2-4cea-4d41-91ff-c57291dd54b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ffaba31c9744c46ac5cfcb38b9e934f, user_id=5afa7c67bc80425a90dddac0e33f8424, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:56.110366 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f7cad2-4cea-4d41-91ff-c57291dd54b4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:56.125047 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1201/2408] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:23:56 2022] PUT /identity/v3/projects/0ffaba31c9744c46ac5cfcb38b9e934f/users/5afa7c67bc80425a90dddac0e33f8424/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:56.136428 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722af81-87a0-4d04-8bea-fc4ae5f58b8d None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:56.136718 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722af81-87a0-4d04-8bea-fc4ae5f58b8d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:56.137031 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722af81-87a0-4d04-8bea-fc4ae5f58b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:56.159945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:56.174615 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-d722af81-87a0-4d04-8bea-fc4ae5f58b8d None None] MFA Rules not processed for user `5afa7c67bc80425a90dddac0e33f8424`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:56.176250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d722af81-87a0-4d04-8bea-fc4ae5f58b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:56.249831 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1208/2409] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:56.258011 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7cad2-4cea-4d41-91ff-c57291dd54b4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:56.310759 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:56.326752 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7cad2-4cea-4d41-91ff-c57291dd54b4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:56.331683 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342dad39-e122-4252-96f7-744b179ad94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:56.332350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342dad39-e122-4252-96f7-744b179ad94a admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:56.332637 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342dad39-e122-4252-96f7-744b179ad94a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:56.332916 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342dad39-e122-4252-96f7-744b179ad94a admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:56.333640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342dad39-e122-4252-96f7-744b179ad94a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:56.335576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342dad39-e122-4252-96f7-744b179ad94a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:56.366861 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1202/2410] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:23:56 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:56.370029 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722af81-87a0-4d04-8bea-fc4ae5f58b8d None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:56.380803 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722af81-87a0-4d04-8bea-fc4ae5f58b8d None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:56.438521 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e44a9-0325-4230-a194-b6cd59520fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:56.439252 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e44a9-0325-4230-a194-b6cd59520fa6 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:56.439536 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e44a9-0325-4230-a194-b6cd59520fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:56.439814 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e44a9-0325-4230-a194-b6cd59520fa6 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:56.440502 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864e44a9-0325-4230-a194-b6cd59520fa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:56.442417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864e44a9-0325-4230-a194-b6cd59520fa6 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:56.464879 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1209/2411] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:23:56 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:56.468832 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342dad39-e122-4252-96f7-744b179ad94a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:56.479423 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342dad39-e122-4252-96f7-744b179ad94a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:56.484141 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed20cb-ec79-4ea0-b125-3388f84ed645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:56.484802 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed20cb-ec79-4ea0-b125-3388f84ed645 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:56.485131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed20cb-ec79-4ea0-b125-3388f84ed645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:56.485348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed20cb-ec79-4ea0-b125-3388f84ed645 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:56.486079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed20cb-ec79-4ea0-b125-3388f84ed645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:56.487745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed20cb-ec79-4ea0-b125-3388f84ed645 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:56.500223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1203/2412] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:23:56.504240 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e44a9-0325-4230-a194-b6cd59520fa6 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:56.515008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e44a9-0325-4230-a194-b6cd59520fa6 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:56.521447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa97e7-dfb3-4069-bef0-a8b500016dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:56.521447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa97e7-dfb3-4069-bef0-a8b500016dd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:56.521447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa97e7-dfb3-4069-bef0-a8b500016dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:56.521447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa97e7-dfb3-4069-bef0-a8b500016dd0 admin admin] PATH_INFO: `/v3/projects/fa2c054d54d14c0ebef458d37a3c1c47/users/2248679fb10a41adaba1f069579c0219/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:56.537122 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa97e7-dfb3-4069-bef0-a8b500016dd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa2c054d54d14c0ebef458d37a3c1c47, user_id=2248679fb10a41adaba1f069579c0219, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:56.539985 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa97e7-dfb3-4069-bef0-a8b500016dd0 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:56.554745 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1210/2413] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:23:56 2022] PUT /identity/v3/projects/fa2c054d54d14c0ebef458d37a3c1c47/users/2248679fb10a41adaba1f069579c0219/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:23:56.559410 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed20cb-ec79-4ea0-b125-3388f84ed645 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:56.569569 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed20cb-ec79-4ea0-b125-3388f84ed645 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:56.635656 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15a149-046e-44a9-9b5e-0590c19c8503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:56.636672 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15a149-046e-44a9-9b5e-0590c19c8503 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:56.637126 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15a149-046e-44a9-9b5e-0590c19c8503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:56.637402 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15a149-046e-44a9-9b5e-0590c19c8503 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:56.638198 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f15a149-046e-44a9-9b5e-0590c19c8503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:56.639962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f15a149-046e-44a9-9b5e-0590c19c8503 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:56.664720 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa97e7-dfb3-4069-bef0-a8b500016dd0 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:56.666848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1204/2414] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:23:56.677777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa97e7-dfb3-4069-bef0-a8b500016dd0 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:56.687720 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc02804-42bd-42bb-8bb2-2b19b6dbb21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:56.688934 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc02804-42bd-42bb-8bb2-2b19b6dbb21c admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:56.689112 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc02804-42bd-42bb-8bb2-2b19b6dbb21c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:56.689517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc02804-42bd-42bb-8bb2-2b19b6dbb21c admin admin] PATH_INFO: `/v3/projects/fa2c054d54d14c0ebef458d37a3c1c47/users/2248679fb10a41adaba1f069579c0219/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:56.691700 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc02804-42bd-42bb-8bb2-2b19b6dbb21c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa2c054d54d14c0ebef458d37a3c1c47, user_id=2248679fb10a41adaba1f069579c0219, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:56.694693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc02804-42bd-42bb-8bb2-2b19b6dbb21c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:56.707649 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1211/2415] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:23:56 2022] PUT /identity/v3/projects/fa2c054d54d14c0ebef458d37a3c1c47/users/2248679fb10a41adaba1f069579c0219/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:23:56.718439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd5445-a444-4251-a346-8830291adcdd None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:56.718942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd5445-a444-4251-a346-8830291adcdd None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:56.719061 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd5445-a444-4251-a346-8830291adcdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:56.749557 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-73bd5445-a444-4251-a346-8830291adcdd None None] MFA Rules not processed for user `2248679fb10a41adaba1f069579c0219`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:56.751149 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-73bd5445-a444-4251-a346-8830291adcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:56.833994 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1205/2416] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:56.848230 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc02804-42bd-42bb-8bb2-2b19b6dbb21c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:56.850847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ffc02804-42bd-42bb-8bb2-2b19b6dbb21c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:56.935066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:56.992796 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc02804-42bd-42bb-8bb2-2b19b6dbb21c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:56.998623 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72da96-edcc-487e-b3f6-e98c8d3d7f68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:57.011012 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72da96-edcc-487e-b3f6-e98c8d3d7f68 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:57.011012 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72da96-edcc-487e-b3f6-e98c8d3d7f68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:57.012952 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72da96-edcc-487e-b3f6-e98c8d3d7f68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:57.032674 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72da96-edcc-487e-b3f6-e98c8d3d7f68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:57.034958 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72da96-edcc-487e-b3f6-e98c8d3d7f68 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:57.047970 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1212/2417] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:56 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:23:57.170745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd5445-a444-4251-a346-8830291adcdd None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:57.172358 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-73bd5445-a444-4251-a346-8830291adcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:57.187902 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd5445-a444-4251-a346-8830291adcdd None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:57.192375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac65387-e0d3-43c3-ab3d-be30432754ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:57.193153 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac65387-e0d3-43c3-ab3d-be30432754ea service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:57.193548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac65387-e0d3-43c3-ab3d-be30432754ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:57.193899 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac65387-e0d3-43c3-ab3d-be30432754ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:57.255916 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:57.273167 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac65387-e0d3-43c3-ab3d-be30432754ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:57.274748 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac65387-e0d3-43c3-ab3d-be30432754ea service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:57.286183 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1206/2418] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:23:57.593197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72da96-edcc-487e-b3f6-e98c8d3d7f68 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:57.668438 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72da96-edcc-487e-b3f6-e98c8d3d7f68 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:57.673177 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48047b8-6451-4650-aea3-19d901cd04eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:57.673951 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48047b8-6451-4650-aea3-19d901cd04eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:57.674455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48047b8-6451-4650-aea3-19d901cd04eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:57.674574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48047b8-6451-4650-aea3-19d901cd04eb admin admin] PATH_INFO: `/v3/users/5afa7c67bc80425a90dddac0e33f8424` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:57.675849 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48047b8-6451-4650-aea3-19d901cd04eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5afa7c67bc80425a90dddac0e33f8424)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:57.677686 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48047b8-6451-4650-aea3-19d901cd04eb admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:57.725872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e48047b8-6451-4650-aea3-19d901cd04eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5afa7c67bc80425a90dddac0e33f8424'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:57.739698 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e48047b8-6451-4650-aea3-19d901cd04eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5afa7c67bc80425a90dddac0e33f8424'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:57.740414 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e48047b8-6451-4650-aea3-19d901cd04eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5afa7c67bc80425a90dddac0e33f8424'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:57.741337 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e48047b8-6451-4650-aea3-19d901cd04eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5afa7c67bc80425a90dddac0e33f8424'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:57.748994 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e48047b8-6451-4650-aea3-19d901cd04eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5afa7c67bc80425a90dddac0e33f8424'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:57.763318 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1213/2419] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:23:57 2022] DELETE /identity/v3/users/5afa7c67bc80425a90dddac0e33f8424 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:57.768316 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76149c3-6bed-47f7-aca1-59dcd141fee6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:57.768834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76149c3-6bed-47f7-aca1-59dcd141fee6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:57.769137 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76149c3-6bed-47f7-aca1-59dcd141fee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:57.788363 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d76149c3-6bed-47f7-aca1-59dcd141fee6 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:57.789682 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d76149c3-6bed-47f7-aca1-59dcd141fee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:57.852462 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1207/2420] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:57.862757 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48047b8-6451-4650-aea3-19d901cd04eb admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:57.864216 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e48047b8-6451-4650-aea3-19d901cd04eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:57.921126 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48047b8-6451-4650-aea3-19d901cd04eb admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:57.924782 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870939a9-eb6e-42fd-b041-86c17b9d8c6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:57.925425 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870939a9-eb6e-42fd-b041-86c17b9d8c6e service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:57.925706 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870939a9-eb6e-42fd-b041-86c17b9d8c6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:57.925981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870939a9-eb6e-42fd-b041-86c17b9d8c6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:57.932311 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:57.936967 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870939a9-eb6e-42fd-b041-86c17b9d8c6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:57.938670 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870939a9-eb6e-42fd-b041-86c17b9d8c6e service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:57.948858 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1214/2421] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:23:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:23:58.338322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76149c3-6bed-47f7-aca1-59dcd141fee6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:58.338322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d76149c3-6bed-47f7-aca1-59dcd141fee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:58.366305 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76149c3-6bed-47f7-aca1-59dcd141fee6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:58.380011 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95079b9a-3983-466f-bd1e-c6fd43ce1966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:58.381319 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95079b9a-3983-466f-bd1e-c6fd43ce1966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:58.381690 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95079b9a-3983-466f-bd1e-c6fd43ce1966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:58.381974 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95079b9a-3983-466f-bd1e-c6fd43ce1966 admin admin] PATH_INFO: `/v3/projects/0ffaba31c9744c46ac5cfcb38b9e934f` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:58.383204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95079b9a-3983-466f-bd1e-c6fd43ce1966 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ffaba31c9744c46ac5cfcb38b9e934f)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:58.385325 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95079b9a-3983-466f-bd1e-c6fd43ce1966 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:58.441794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-95079b9a-3983-466f-bd1e-c6fd43ce1966 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ffaba31c9744c46ac5cfcb38b9e934f', 'request_id': 'req-95079b9a-3983-466f-bd1e-c6fd43ce1966'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:58.452984 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-95079b9a-3983-466f-bd1e-c6fd43ce1966 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ffaba31c9744c46ac5cfcb38b9e934f', 'request_id': 'req-95079b9a-3983-466f-bd1e-c6fd43ce1966'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:58.461014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-95079b9a-3983-466f-bd1e-c6fd43ce1966 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ffaba31c9744c46ac5cfcb38b9e934f', 'request_id': 'req-95079b9a-3983-466f-bd1e-c6fd43ce1966'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:58.473944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-95079b9a-3983-466f-bd1e-c6fd43ce1966 admin admin] The token cache is being invalidate because project 0ffaba31c9744c46ac5cfcb38b9e934f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:23:58.474275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-95079b9a-3983-466f-bd1e-c6fd43ce1966 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:58.474904 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-95079b9a-3983-466f-bd1e-c6fd43ce1966 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:58.476701 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1208/2422] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:23:58 2022] DELETE /identity/v3/projects/0ffaba31c9744c46ac5cfcb38b9e934f => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:58.480922 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870939a9-eb6e-42fd-b041-86c17b9d8c6e service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:58.493365 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870939a9-eb6e-42fd-b041-86c17b9d8c6e service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:58.560378 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5d9e6-c9e3-48a7-90cf-5beb43073727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:58.561164 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5d9e6-c9e3-48a7-90cf-5beb43073727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:58.561454 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5d9e6-c9e3-48a7-90cf-5beb43073727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:58.561733 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5d9e6-c9e3-48a7-90cf-5beb43073727 admin admin] PATH_INFO: `/v3/users/2248679fb10a41adaba1f069579c0219` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:58.562928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce5d9e6-c9e3-48a7-90cf-5beb43073727 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2248679fb10a41adaba1f069579c0219)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:58.566298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce5d9e6-c9e3-48a7-90cf-5beb43073727 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:58.615907 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3ce5d9e6-c9e3-48a7-90cf-5beb43073727 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2248679fb10a41adaba1f069579c0219'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:58.632358 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3ce5d9e6-c9e3-48a7-90cf-5beb43073727 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2248679fb10a41adaba1f069579c0219'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:58.638983 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3ce5d9e6-c9e3-48a7-90cf-5beb43073727 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2248679fb10a41adaba1f069579c0219'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:58.639688 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3ce5d9e6-c9e3-48a7-90cf-5beb43073727 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2248679fb10a41adaba1f069579c0219'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:58.654525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3ce5d9e6-c9e3-48a7-90cf-5beb43073727 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2248679fb10a41adaba1f069579c0219'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:58.669139 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1215/2423] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:23:58 2022] DELETE /identity/v3/users/2248679fb10a41adaba1f069579c0219 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:58.943075 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95079b9a-3983-466f-bd1e-c6fd43ce1966 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:58.944607 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-95079b9a-3983-466f-bd1e-c6fd43ce1966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:58.966025 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95079b9a-3983-466f-bd1e-c6fd43ce1966 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:59.026071 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7f371-0c5e-4683-9033-d6f783577e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:59.027167 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7f371-0c5e-4683-9033-d6f783577e90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:59.027955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7f371-0c5e-4683-9033-d6f783577e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:59.027955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7f371-0c5e-4683-9033-d6f783577e90 admin admin] PATH_INFO: `/v3/projects/fa2c054d54d14c0ebef458d37a3c1c47` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:59.028931 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd7f371-0c5e-4683-9033-d6f783577e90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa2c054d54d14c0ebef458d37a3c1c47)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:59.030946 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd7f371-0c5e-4683-9033-d6f783577e90 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:59.042047 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:59.078360 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0fd7f371-0c5e-4683-9033-d6f783577e90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa2c054d54d14c0ebef458d37a3c1c47', 'request_id': 'req-0fd7f371-0c5e-4683-9033-d6f783577e90'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:59.085270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0fd7f371-0c5e-4683-9033-d6f783577e90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa2c054d54d14c0ebef458d37a3c1c47', 'request_id': 'req-0fd7f371-0c5e-4683-9033-d6f783577e90'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:59.090754 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0fd7f371-0c5e-4683-9033-d6f783577e90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa2c054d54d14c0ebef458d37a3c1c47', 'request_id': 'req-0fd7f371-0c5e-4683-9033-d6f783577e90'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:59.101757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0fd7f371-0c5e-4683-9033-d6f783577e90 admin admin] The token cache is being invalidate because project fa2c054d54d14c0ebef458d37a3c1c47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:23:59.102043 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0fd7f371-0c5e-4683-9033-d6f783577e90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:59.102674 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0fd7f371-0c5e-4683-9033-d6f783577e90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:23:59.104736 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1209/2424] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:23:58 2022] DELETE /identity/v3/projects/fa2c054d54d14c0ebef458d37a3c1c47 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:23:59.109615 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e4daa-3012-45d5-884a-4ebf65d200a3 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:59.110104 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e4daa-3012-45d5-884a-4ebf65d200a3 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:59.110292 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e4daa-3012-45d5-884a-4ebf65d200a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:59.129025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-580e4daa-3012-45d5-884a-4ebf65d200a3 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:59.130345 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-580e4daa-3012-45d5-884a-4ebf65d200a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:59.192193 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1216/2425] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:59.200364 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5b9e5-945f-4c6b-93c9-707e2a0dd7ef None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:59.200769 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5b9e5-945f-4c6b-93c9-707e2a0dd7ef None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:59.200917 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5b9e5-945f-4c6b-93c9-707e2a0dd7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:59.220387 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1ab5b9e5-945f-4c6b-93c9-707e2a0dd7ef None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:59.221776 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1ab5b9e5-945f-4c6b-93c9-707e2a0dd7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:59.233308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1210/2426] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:59.235565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e4daa-3012-45d5-884a-4ebf65d200a3 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:59.247155 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e4daa-3012-45d5-884a-4ebf65d200a3 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:59.253993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85957c-e159-4545-8423-df677681d144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:59.255255 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85957c-e159-4545-8423-df677681d144 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:59.255255 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85957c-e159-4545-8423-df677681d144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:59.255255 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85957c-e159-4545-8423-df677681d144 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:59.256021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85957c-e159-4545-8423-df677681d144 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:59.256328 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85957c-e159-4545-8423-df677681d144 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:59.258059 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85957c-e159-4545-8423-df677681d144 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:59.271950 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:59.276533 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1217/2427] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:23:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:59.280980 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab5b9e5-945f-4c6b-93c9-707e2a0dd7ef None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:59.290302 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab5b9e5-945f-4c6b-93c9-707e2a0dd7ef None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:59.295340 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020c9b7-4a24-471b-9d75-42056924ad42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:59.296078 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020c9b7-4a24-471b-9d75-42056924ad42 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:59.296431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020c9b7-4a24-471b-9d75-42056924ad42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:59.297342 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020c9b7-4a24-471b-9d75-42056924ad42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:59.297653 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d020c9b7-4a24-471b-9d75-42056924ad42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:59.299671 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d020c9b7-4a24-471b-9d75-42056924ad42 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:59.329812 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1211/2428] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:23:59 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:59.333989 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85957c-e159-4545-8423-df677681d144 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:59.344905 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85957c-e159-4545-8423-df677681d144 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:59.401038 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087cffa6-1206-4e1a-9e1e-48567b96c361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:59.401705 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cffa6-1206-4e1a-9e1e-48567b96c361 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:59.401984 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cffa6-1206-4e1a-9e1e-48567b96c361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:59.402257 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cffa6-1206-4e1a-9e1e-48567b96c361 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:59.402982 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087cffa6-1206-4e1a-9e1e-48567b96c361 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:59.404928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087cffa6-1206-4e1a-9e1e-48567b96c361 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:59.425551 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1218/2429] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:23:59 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:59.430272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020c9b7-4a24-471b-9d75-42056924ad42 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:59.441463 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020c9b7-4a24-471b-9d75-42056924ad42 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:59.446235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bfaea2-c9b1-4b06-a726-cbcc563471a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:59.446895 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfaea2-c9b1-4b06-a726-cbcc563471a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:59.448019 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfaea2-c9b1-4b06-a726-cbcc563471a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:59.448458 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfaea2-c9b1-4b06-a726-cbcc563471a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:59.449187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bfaea2-c9b1-4b06-a726-cbcc563471a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:59.450798 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bfaea2-c9b1-4b06-a726-cbcc563471a1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:59.461273 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1212/2430] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:23:59.465274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087cffa6-1206-4e1a-9e1e-48567b96c361 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:59.475467 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087cffa6-1206-4e1a-9e1e-48567b96c361 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:59.480220 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41743400-060b-414f-8185-f0096b498501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:59.480878 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41743400-060b-414f-8185-f0096b498501 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:59.481165 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41743400-060b-414f-8185-f0096b498501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:59.481449 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41743400-060b-414f-8185-f0096b498501 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:59.482174 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41743400-060b-414f-8185-f0096b498501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:59.483790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41743400-060b-414f-8185-f0096b498501 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:59.494577 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1219/2431] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:23:59.498632 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bfaea2-c9b1-4b06-a726-cbcc563471a1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:59.508500 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bfaea2-c9b1-4b06-a726-cbcc563471a1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:59.513781 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166d48e-a1fb-44ec-8207-a92452325c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:59.514651 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166d48e-a1fb-44ec-8207-a92452325c2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:59.514931 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166d48e-a1fb-44ec-8207-a92452325c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:59.515204 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166d48e-a1fb-44ec-8207-a92452325c2d admin admin] PATH_INFO: `/v3/projects/93f8bc0d89c142a591098fa617dbbc78/users/a662be4b67aa482389d2209ad5476893/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:59.529490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8166d48e-a1fb-44ec-8207-a92452325c2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=93f8bc0d89c142a591098fa617dbbc78, user_id=a662be4b67aa482389d2209ad5476893, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:59.532156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8166d48e-a1fb-44ec-8207-a92452325c2d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:59.545319 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1213/2432] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:23:59 2022] PUT /identity/v3/projects/93f8bc0d89c142a591098fa617dbbc78/users/a662be4b67aa482389d2209ad5476893/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:59.556315 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4348d4-c951-4ca2-8ffc-f18e2a377a3a None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:59.557409 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4348d4-c951-4ca2-8ffc-f18e2a377a3a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:59.557751 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4348d4-c951-4ca2-8ffc-f18e2a377a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:59.583352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6d4348d4-c951-4ca2-8ffc-f18e2a377a3a None None] MFA Rules not processed for user `a662be4b67aa482389d2209ad5476893`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:59.584474 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6d4348d4-c951-4ca2-8ffc-f18e2a377a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:59.648069 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1220/2433] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:23:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:23:59.652587 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166d48e-a1fb-44ec-8207-a92452325c2d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:59.719953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166d48e-a1fb-44ec-8207-a92452325c2d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:59.724888 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834700e-2671-423d-81ef-b12b7701e38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:59.725565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834700e-2671-423d-81ef-b12b7701e38e admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:59.725846 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834700e-2671-423d-81ef-b12b7701e38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:59.726127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834700e-2671-423d-81ef-b12b7701e38e admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:59.726826 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834700e-2671-423d-81ef-b12b7701e38e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:59.729452 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834700e-2671-423d-81ef-b12b7701e38e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:59.755860 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1214/2434] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:23:59 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:59.759777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4348d4-c951-4ca2-8ffc-f18e2a377a3a None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:59.770444 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4348d4-c951-4ca2-8ffc-f18e2a377a3a None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:59.828501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e96bb8-cdce-40a2-a930-37281f6e8f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:59.829269 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e96bb8-cdce-40a2-a930-37281f6e8f4f admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:59.829632 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e96bb8-cdce-40a2-a930-37281f6e8f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:59.829989 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e96bb8-cdce-40a2-a930-37281f6e8f4f admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:59.830787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e96bb8-cdce-40a2-a930-37281f6e8f4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:59.832819 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e96bb8-cdce-40a2-a930-37281f6e8f4f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:59.849100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:59.859044 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1221/2435] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:23:59 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:59.862593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834700e-2671-423d-81ef-b12b7701e38e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:59.874440 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834700e-2671-423d-81ef-b12b7701e38e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:59.879430 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141dbc72-0828-4847-98a1-b5c03ff94df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:59.880102 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141dbc72-0828-4847-98a1-b5c03ff94df3 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:59.880383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141dbc72-0828-4847-98a1-b5c03ff94df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:59.880660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141dbc72-0828-4847-98a1-b5c03ff94df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:59.881433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141dbc72-0828-4847-98a1-b5c03ff94df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:59.883028 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141dbc72-0828-4847-98a1-b5c03ff94df3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:59.894901 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1215/2436] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:23:59.898896 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e96bb8-cdce-40a2-a930-37281f6e8f4f admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:59.910654 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e96bb8-cdce-40a2-a930-37281f6e8f4f admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:59.915351 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabeb509-cc96-4693-bd63-4ef9ad14d58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:59.917120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabeb509-cc96-4693-bd63-4ef9ad14d58f admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:59.917511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabeb509-cc96-4693-bd63-4ef9ad14d58f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:59.917839 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabeb509-cc96-4693-bd63-4ef9ad14d58f admin admin] PATH_INFO: `/v3/projects/0011ed2f703c48c7a1310b143cd67a2f/users/e205ae9954eb4278a5858a7fa4cca2a8/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:59.933207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabeb509-cc96-4693-bd63-4ef9ad14d58f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0011ed2f703c48c7a1310b143cd67a2f, user_id=e205ae9954eb4278a5858a7fa4cca2a8, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:59.935956 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabeb509-cc96-4693-bd63-4ef9ad14d58f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:59.949224 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1222/2437] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:23:59 2022] PUT /identity/v3/projects/0011ed2f703c48c7a1310b143cd67a2f/users/e205ae9954eb4278a5858a7fa4cca2a8/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:59.953260 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141dbc72-0828-4847-98a1-b5c03ff94df3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:59.963884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141dbc72-0828-4847-98a1-b5c03ff94df3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:59.968638 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:00.023543 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43f47fd-4d23-44f9-925e-d86db462fdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:00.024213 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f47fd-4d23-44f9-925e-d86db462fdff admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:00.024496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f47fd-4d23-44f9-925e-d86db462fdff admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:00.024777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f47fd-4d23-44f9-925e-d86db462fdff admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:00.025650 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43f47fd-4d23-44f9-925e-d86db462fdff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:00.027268 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43f47fd-4d23-44f9-925e-d86db462fdff admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:00.036822 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1216/2438] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:24:00.041189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabeb509-cc96-4693-bd63-4ef9ad14d58f admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:00.051542 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabeb509-cc96-4693-bd63-4ef9ad14d58f admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:00.056939 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d66bb-86ec-4868-9fec-1b80dcf759d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:00.058068 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d66bb-86ec-4868-9fec-1b80dcf759d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:00.058349 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d66bb-86ec-4868-9fec-1b80dcf759d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:00.058626 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d66bb-86ec-4868-9fec-1b80dcf759d8 admin admin] PATH_INFO: `/v3/projects/0011ed2f703c48c7a1310b143cd67a2f/users/e205ae9954eb4278a5858a7fa4cca2a8/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:00.060664 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d66bb-86ec-4868-9fec-1b80dcf759d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0011ed2f703c48c7a1310b143cd67a2f, user_id=e205ae9954eb4278a5858a7fa4cca2a8, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:00.063326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d66bb-86ec-4868-9fec-1b80dcf759d8 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:00.074237 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1223/2439] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:24:00 2022] PUT /identity/v3/projects/0011ed2f703c48c7a1310b143cd67a2f/users/e205ae9954eb4278a5858a7fa4cca2a8/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:24:00.084044 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06568951-6f5b-4df6-9a0f-dfb96272a542 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:00.084332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06568951-6f5b-4df6-9a0f-dfb96272a542 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:00.084613 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06568951-6f5b-4df6-9a0f-dfb96272a542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:00.110787 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-06568951-6f5b-4df6-9a0f-dfb96272a542 None None] MFA Rules not processed for user `e205ae9954eb4278a5858a7fa4cca2a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:24:00.111949 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-06568951-6f5b-4df6-9a0f-dfb96272a542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:00.179232 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1217/2440] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:24:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:24:00.190235 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d66bb-86ec-4868-9fec-1b80dcf759d8 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:00.191923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f26d66bb-86ec-4868-9fec-1b80dcf759d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:00.252504 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d66bb-86ec-4868-9fec-1b80dcf759d8 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:00.256314 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c66d24-3aac-41bf-84f0-1be63c9ff80a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:00.257002 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c66d24-3aac-41bf-84f0-1be63c9ff80a service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:00.257289 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c66d24-3aac-41bf-84f0-1be63c9ff80a service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:00.257570 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c66d24-3aac-41bf-84f0-1be63c9ff80a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:00.454748 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c66d24-3aac-41bf-84f0-1be63c9ff80a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:00.456214 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c66d24-3aac-41bf-84f0-1be63c9ff80a service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:00.468243 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1224/2441] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:24:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:24:05.002689 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06568951-6f5b-4df6-9a0f-dfb96272a542 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:05.005203 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-06568951-6f5b-4df6-9a0f-dfb96272a542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:05.026035 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06568951-6f5b-4df6-9a0f-dfb96272a542 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:05.031931 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847458b6-5d84-4ef2-9943-20ddeaec056a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:05.032469 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847458b6-5d84-4ef2-9943-20ddeaec056a service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:05.032750 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847458b6-5d84-4ef2-9943-20ddeaec056a service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:05.033263 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847458b6-5d84-4ef2-9943-20ddeaec056a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:05.071852 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847458b6-5d84-4ef2-9943-20ddeaec056a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:05.073792 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847458b6-5d84-4ef2-9943-20ddeaec056a service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:05.086352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1218/2442] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:24:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:24:07.207020 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c66d24-3aac-41bf-84f0-1be63c9ff80a service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:07.210319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-71c66d24-3aac-41bf-84f0-1be63c9ff80a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:07.295484 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c66d24-3aac-41bf-84f0-1be63c9ff80a service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:07.300313 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132866bf-dd60-4132-ba3d-dd964c5862fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:07.301296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132866bf-dd60-4132-ba3d-dd964c5862fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:07.301580 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132866bf-dd60-4132-ba3d-dd964c5862fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:07.301859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132866bf-dd60-4132-ba3d-dd964c5862fe admin admin] PATH_INFO: `/v3/users/06c1b76b1a5740d1945bbec2cc25a902` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:07.303579 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132866bf-dd60-4132-ba3d-dd964c5862fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=06c1b76b1a5740d1945bbec2cc25a902)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:07.305207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132866bf-dd60-4132-ba3d-dd964c5862fe admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:07.324801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:07.372599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-132866bf-dd60-4132-ba3d-dd964c5862fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06c1b76b1a5740d1945bbec2cc25a902'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:07.388138 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-132866bf-dd60-4132-ba3d-dd964c5862fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06c1b76b1a5740d1945bbec2cc25a902'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:07.389456 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-132866bf-dd60-4132-ba3d-dd964c5862fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06c1b76b1a5740d1945bbec2cc25a902'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:07.390240 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-132866bf-dd60-4132-ba3d-dd964c5862fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06c1b76b1a5740d1945bbec2cc25a902'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:07.398588 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-132866bf-dd60-4132-ba3d-dd964c5862fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06c1b76b1a5740d1945bbec2cc25a902'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:07.421282 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1225/2443] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:24:07 2022] DELETE /identity/v3/users/06c1b76b1a5740d1945bbec2cc25a902 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:24:07.425670 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6202898-2569-4de8-b035-2a806ee77a41 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:07.426143 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6202898-2569-4de8-b035-2a806ee77a41 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:07.426360 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6202898-2569-4de8-b035-2a806ee77a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:07.447202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b6202898-2569-4de8-b035-2a806ee77a41 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:24:07.448734 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b6202898-2569-4de8-b035-2a806ee77a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:07.526841 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1219/2444] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:24:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:24:07.561580 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132866bf-dd60-4132-ba3d-dd964c5862fe admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:07.563032 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-132866bf-dd60-4132-ba3d-dd964c5862fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:07.656550 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132866bf-dd60-4132-ba3d-dd964c5862fe admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:07.660850 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980453c-83da-426b-95b7-32102d142c8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:07.661857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980453c-83da-426b-95b7-32102d142c8f service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:07.661857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980453c-83da-426b-95b7-32102d142c8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:07.662122 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980453c-83da-426b-95b7-32102d142c8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:07.684580 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980453c-83da-426b-95b7-32102d142c8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:07.686757 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980453c-83da-426b-95b7-32102d142c8f service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:07.698952 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1226/2445] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:24:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:24:08.481037 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6202898-2569-4de8-b035-2a806ee77a41 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:08.483346 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b6202898-2569-4de8-b035-2a806ee77a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:08.493254 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef149b-e283-4983-a467-ea47e589b47b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:08.493713 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef149b-e283-4983-a467-ea47e589b47b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:08.493811 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef149b-e283-4983-a467-ea47e589b47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:08.504796 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6202898-2569-4de8-b035-2a806ee77a41 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:08.512796 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44fadb-c9fa-4d37-a755-f70da0f78511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:08.514167 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44fadb-c9fa-4d37-a755-f70da0f78511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:08.518475 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-44ef149b-e283-4983-a467-ea47e589b47b None None] MFA Rules not processed for user `d20025ab12f4450da20c8e41b931b5fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:24:08.518475 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-44ef149b-e283-4983-a467-ea47e589b47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:08.519504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44fadb-c9fa-4d37-a755-f70da0f78511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:08.519844 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44fadb-c9fa-4d37-a755-f70da0f78511 admin admin] PATH_INFO: `/v3/projects/7e1e6566bc3345a69293b67c3edf2677` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:08.521531 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44fadb-c9fa-4d37-a755-f70da0f78511 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e1e6566bc3345a69293b67c3edf2677)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:08.523506 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44fadb-c9fa-4d37-a755-f70da0f78511 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:08.553830 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:08.603534 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1227/2446] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:24:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:24:08.616540 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1c44fadb-c9fa-4d37-a755-f70da0f78511 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e1e6566bc3345a69293b67c3edf2677', 'request_id': 'req-1c44fadb-c9fa-4d37-a755-f70da0f78511'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:08.639091 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1c44fadb-c9fa-4d37-a755-f70da0f78511 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e1e6566bc3345a69293b67c3edf2677', 'request_id': 'req-1c44fadb-c9fa-4d37-a755-f70da0f78511'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:08.654433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1c44fadb-c9fa-4d37-a755-f70da0f78511 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e1e6566bc3345a69293b67c3edf2677', 'request_id': 'req-1c44fadb-c9fa-4d37-a755-f70da0f78511'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:08.667511 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1c44fadb-c9fa-4d37-a755-f70da0f78511 admin admin] The token cache is being invalidate because project 7e1e6566bc3345a69293b67c3edf2677 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:24:08.667794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1c44fadb-c9fa-4d37-a755-f70da0f78511 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:08.668738 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1c44fadb-c9fa-4d37-a755-f70da0f78511 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:08.671381 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1220/2447] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:24:08 2022] DELETE /identity/v3/projects/7e1e6566bc3345a69293b67c3edf2677 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:24:10.254290 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef149b-e283-4983-a467-ea47e589b47b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:10.255809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-44ef149b-e283-4983-a467-ea47e589b47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:10.269368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef149b-e283-4983-a467-ea47e589b47b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:10.311815 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:10.329821 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94da2f7-2820-47fd-bcc1-942bdc47870e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:10.330699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94da2f7-2820-47fd-bcc1-942bdc47870e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:10.330983 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94da2f7-2820-47fd-bcc1-942bdc47870e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:10.331291 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94da2f7-2820-47fd-bcc1-942bdc47870e admin admin] PATH_INFO: `/v3/users/a662be4b67aa482389d2209ad5476893` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:10.332474 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94da2f7-2820-47fd-bcc1-942bdc47870e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a662be4b67aa482389d2209ad5476893)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:10.334403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94da2f7-2820-47fd-bcc1-942bdc47870e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:10.382131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b94da2f7-2820-47fd-bcc1-942bdc47870e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a662be4b67aa482389d2209ad5476893'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:10.394045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b94da2f7-2820-47fd-bcc1-942bdc47870e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a662be4b67aa482389d2209ad5476893'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:10.394718 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b94da2f7-2820-47fd-bcc1-942bdc47870e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a662be4b67aa482389d2209ad5476893'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:10.395359 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b94da2f7-2820-47fd-bcc1-942bdc47870e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a662be4b67aa482389d2209ad5476893'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:10.402963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b94da2f7-2820-47fd-bcc1-942bdc47870e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a662be4b67aa482389d2209ad5476893'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:10.417504 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1228/2448] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:24:10 2022] DELETE /identity/v3/users/a662be4b67aa482389d2209ad5476893 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:24:10.422043 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31cb86-12d2-4a3a-a44b-be7c98b6a17d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:10.422747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31cb86-12d2-4a3a-a44b-be7c98b6a17d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:10.422747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31cb86-12d2-4a3a-a44b-be7c98b6a17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:10.441613 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ef31cb86-12d2-4a3a-a44b-be7c98b6a17d None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:24:10.443028 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ef31cb86-12d2-4a3a-a44b-be7c98b6a17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:10.543280 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1221/2449] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:24:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:24:10.553034 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94da2f7-2820-47fd-bcc1-942bdc47870e admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:10.554491 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b94da2f7-2820-47fd-bcc1-942bdc47870e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:10.661466 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94da2f7-2820-47fd-bcc1-942bdc47870e admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:10.669061 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090196e-3e19-42d2-b07a-258c4295db70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:10.670813 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090196e-3e19-42d2-b07a-258c4295db70 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:10.671293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090196e-3e19-42d2-b07a-258c4295db70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:10.671580 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090196e-3e19-42d2-b07a-258c4295db70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:10.685055 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090196e-3e19-42d2-b07a-258c4295db70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:10.686985 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090196e-3e19-42d2-b07a-258c4295db70 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:10.697879 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1229/2450] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:24:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:24:11.308715 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31cb86-12d2-4a3a-a44b-be7c98b6a17d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:11.310280 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ef31cb86-12d2-4a3a-a44b-be7c98b6a17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:11.322648 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31cb86-12d2-4a3a-a44b-be7c98b6a17d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:11.329550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfad38-0069-4698-9dd4-6579fe1dc84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:11.330696 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfad38-0069-4698-9dd4-6579fe1dc84d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:11.331196 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfad38-0069-4698-9dd4-6579fe1dc84d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:11.331742 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfad38-0069-4698-9dd4-6579fe1dc84d admin admin] PATH_INFO: `/v3/projects/93f8bc0d89c142a591098fa617dbbc78` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:11.333310 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfad38-0069-4698-9dd4-6579fe1dc84d admin admin] RBAC: Authorizing `identity:delete_project(project_id=93f8bc0d89c142a591098fa617dbbc78)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:11.335787 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfad38-0069-4698-9dd4-6579fe1dc84d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:11.431168 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a7dfad38-0069-4698-9dd4-6579fe1dc84d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93f8bc0d89c142a591098fa617dbbc78', 'request_id': 'req-a7dfad38-0069-4698-9dd4-6579fe1dc84d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:11.442942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a7dfad38-0069-4698-9dd4-6579fe1dc84d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93f8bc0d89c142a591098fa617dbbc78', 'request_id': 'req-a7dfad38-0069-4698-9dd4-6579fe1dc84d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:11.452005 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a7dfad38-0069-4698-9dd4-6579fe1dc84d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93f8bc0d89c142a591098fa617dbbc78', 'request_id': 'req-a7dfad38-0069-4698-9dd4-6579fe1dc84d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:11.468357 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a7dfad38-0069-4698-9dd4-6579fe1dc84d admin admin] The token cache is being invalidate because project 93f8bc0d89c142a591098fa617dbbc78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:24:11.468717 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a7dfad38-0069-4698-9dd4-6579fe1dc84d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:11.469420 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a7dfad38-0069-4698-9dd4-6579fe1dc84d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:11.471439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1222/2451] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:24:11 2022] DELETE /identity/v3/projects/93f8bc0d89c142a591098fa617dbbc78 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:24:11.476083 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090196e-3e19-42d2-b07a-258c4295db70 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:11.491029 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090196e-3e19-42d2-b07a-258c4295db70 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:11.594777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374e491-7aff-4c42-a6fc-38020c3aa7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:11.596171 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374e491-7aff-4c42-a6fc-38020c3aa7dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:11.596739 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374e491-7aff-4c42-a6fc-38020c3aa7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:11.597478 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374e491-7aff-4c42-a6fc-38020c3aa7dc admin admin] PATH_INFO: `/v3/users/e205ae9954eb4278a5858a7fa4cca2a8` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:11.604202 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5374e491-7aff-4c42-a6fc-38020c3aa7dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e205ae9954eb4278a5858a7fa4cca2a8)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:11.606641 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5374e491-7aff-4c42-a6fc-38020c3aa7dc admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:11.622673 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:11.702873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-5374e491-7aff-4c42-a6fc-38020c3aa7dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e205ae9954eb4278a5858a7fa4cca2a8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:11.733867 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-5374e491-7aff-4c42-a6fc-38020c3aa7dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e205ae9954eb4278a5858a7fa4cca2a8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:11.735072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-5374e491-7aff-4c42-a6fc-38020c3aa7dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e205ae9954eb4278a5858a7fa4cca2a8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:11.736215 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-5374e491-7aff-4c42-a6fc-38020c3aa7dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e205ae9954eb4278a5858a7fa4cca2a8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:11.744823 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-5374e491-7aff-4c42-a6fc-38020c3aa7dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e205ae9954eb4278a5858a7fa4cca2a8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:11.770588 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1230/2452] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:24:11 2022] DELETE /identity/v3/users/e205ae9954eb4278a5858a7fa4cca2a8 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:24:13.022075 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfad38-0069-4698-9dd4-6579fe1dc84d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:13.023587 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a7dfad38-0069-4698-9dd4-6579fe1dc84d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:13.047087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfad38-0069-4698-9dd4-6579fe1dc84d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:13.121091 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:13.196154 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41072c38-0f37-41da-ade8-79dd46cb8841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:13.199705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41072c38-0f37-41da-ade8-79dd46cb8841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:13.201814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41072c38-0f37-41da-ade8-79dd46cb8841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:13.202292 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41072c38-0f37-41da-ade8-79dd46cb8841 admin admin] PATH_INFO: `/v3/projects/0011ed2f703c48c7a1310b143cd67a2f` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:13.204189 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41072c38-0f37-41da-ade8-79dd46cb8841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0011ed2f703c48c7a1310b143cd67a2f)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:13.206504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41072c38-0f37-41da-ade8-79dd46cb8841 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:13.332864 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-41072c38-0f37-41da-ade8-79dd46cb8841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0011ed2f703c48c7a1310b143cd67a2f', 'request_id': 'req-41072c38-0f37-41da-ade8-79dd46cb8841'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:13.351757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-41072c38-0f37-41da-ade8-79dd46cb8841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0011ed2f703c48c7a1310b143cd67a2f', 'request_id': 'req-41072c38-0f37-41da-ade8-79dd46cb8841'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:13.361200 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-41072c38-0f37-41da-ade8-79dd46cb8841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0011ed2f703c48c7a1310b143cd67a2f', 'request_id': 'req-41072c38-0f37-41da-ade8-79dd46cb8841'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:13.376237 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-41072c38-0f37-41da-ade8-79dd46cb8841 admin admin] The token cache is being invalidate because project 0011ed2f703c48c7a1310b143cd67a2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:24:13.376614 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-41072c38-0f37-41da-ade8-79dd46cb8841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:13.377304 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-41072c38-0f37-41da-ade8-79dd46cb8841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:13.379893 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1223/2453] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:24:13 2022] DELETE /identity/v3/projects/0011ed2f703c48c7a1310b143cd67a2f => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:24:13.387407 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a9ab5-cb42-47a9-a02c-416802a36eb4 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:13.387918 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a9ab5-cb42-47a9-a02c-416802a36eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:13.388781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a9ab5-cb42-47a9-a02c-416802a36eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:13.420098 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-b60a9ab5-cb42-47a9-a02c-416802a36eb4 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:24:13.421415 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b60a9ab5-cb42-47a9-a02c-416802a36eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:13.520354 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1231/2454] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:24:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:24:13.529797 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf9b49-0195-410a-a652-b81073db0397 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:13.530343 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf9b49-0195-410a-a652-b81073db0397 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:13.530722 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf9b49-0195-410a-a652-b81073db0397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:13.567478 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-dcaf9b49-0195-410a-a652-b81073db0397 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:24:13.568741 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dcaf9b49-0195-410a-a652-b81073db0397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:13.588826 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1224/2455] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:24:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:24:13.593734 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a9ab5-cb42-47a9-a02c-416802a36eb4 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:13.612484 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a9ab5-cb42-47a9-a02c-416802a36eb4 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:13.617891 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027e293-61cd-44c9-b800-778f86ff6b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:13.618696 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027e293-61cd-44c9-b800-778f86ff6b23 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:13.619005 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027e293-61cd-44c9-b800-778f86ff6b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:13.619484 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027e293-61cd-44c9-b800-778f86ff6b23 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:13.621087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f027e293-61cd-44c9-b800-778f86ff6b23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:24:13.621294 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f027e293-61cd-44c9-b800-778f86ff6b23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:13.622928 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f027e293-61cd-44c9-b800-778f86ff6b23 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:13.639272 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1232/2456] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:24:13.643170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf9b49-0195-410a-a652-b81073db0397 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:13.657641 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf9b49-0195-410a-a652-b81073db0397 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:13.664824 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf0803-0e2e-42f7-a81d-61c2d03dfb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:13.665524 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf0803-0e2e-42f7-a81d-61c2d03dfb22 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:13.666279 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf0803-0e2e-42f7-a81d-61c2d03dfb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:13.666571 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf0803-0e2e-42f7-a81d-61c2d03dfb22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:13.667358 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf0803-0e2e-42f7-a81d-61c2d03dfb22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:13.670020 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf0803-0e2e-42f7-a81d-61c2d03dfb22 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:13.730599 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1225/2457] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:24:13 2022] POST /identity/v3/projects => generated 373 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:24:13.731819 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027e293-61cd-44c9-b800-778f86ff6b23 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:13.756584 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027e293-61cd-44c9-b800-778f86ff6b23 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:13.852367 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd501c26-2848-4053-ac36-ffc70d19cc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:13.853165 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd501c26-2848-4053-ac36-ffc70d19cc65 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:13.854024 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd501c26-2848-4053-ac36-ffc70d19cc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:13.854915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd501c26-2848-4053-ac36-ffc70d19cc65 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:13.855741 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd501c26-2848-4053-ac36-ffc70d19cc65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:13.857832 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd501c26-2848-4053-ac36-ffc70d19cc65 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:13.880339 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:13.898226 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1233/2458] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:24:13 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:24:13.902559 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf0803-0e2e-42f7-a81d-61c2d03dfb22 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:13.914426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf0803-0e2e-42f7-a81d-61c2d03dfb22 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:13.923628 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c006b9-69af-4aa5-b5e1-6df9a9090164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:13.924635 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c006b9-69af-4aa5-b5e1-6df9a9090164 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:13.924728 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c006b9-69af-4aa5-b5e1-6df9a9090164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:13.924954 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c006b9-69af-4aa5-b5e1-6df9a9090164 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:13.926056 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c006b9-69af-4aa5-b5e1-6df9a9090164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:13.927359 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c006b9-69af-4aa5-b5e1-6df9a9090164 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:13.941710 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1226/2459] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:24:13.943292 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd501c26-2848-4053-ac36-ffc70d19cc65 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:13.954701 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd501c26-2848-4053-ac36-ffc70d19cc65 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:13.961064 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380bde5e-8315-4088-8159-ec879469d23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:13.961745 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bde5e-8315-4088-8159-ec879469d23f admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:13.962031 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bde5e-8315-4088-8159-ec879469d23f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:13.962312 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bde5e-8315-4088-8159-ec879469d23f admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:13.963049 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380bde5e-8315-4088-8159-ec879469d23f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:13.965183 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380bde5e-8315-4088-8159-ec879469d23f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:13.977162 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1234/2460] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:24:13.981900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c006b9-69af-4aa5-b5e1-6df9a9090164 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:13.995268 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c006b9-69af-4aa5-b5e1-6df9a9090164 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:14.007005 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577db037-48bb-496b-9a59-f03c814e5135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:14.008149 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577db037-48bb-496b-9a59-f03c814e5135 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:14.008377 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577db037-48bb-496b-9a59-f03c814e5135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:14.008794 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577db037-48bb-496b-9a59-f03c814e5135 admin admin] PATH_INFO: `/v3/projects/fcddbc7dcdc64b95ae7a63348254cba9/users/4fafbf3aec7a48e6b67839b32cc01964/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:14.030153 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577db037-48bb-496b-9a59-f03c814e5135 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcddbc7dcdc64b95ae7a63348254cba9, user_id=4fafbf3aec7a48e6b67839b32cc01964, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:14.032893 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577db037-48bb-496b-9a59-f03c814e5135 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:14.051090 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1227/2461] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:24:13 2022] PUT /identity/v3/projects/fcddbc7dcdc64b95ae7a63348254cba9/users/4fafbf3aec7a48e6b67839b32cc01964/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:24:14.062706 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711db383-59b2-4b34-95f8-9d5a884e1b59 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:14.063008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711db383-59b2-4b34-95f8-9d5a884e1b59 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:14.063319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711db383-59b2-4b34-95f8-9d5a884e1b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:14.094616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-711db383-59b2-4b34-95f8-9d5a884e1b59 None None] MFA Rules not processed for user `4fafbf3aec7a48e6b67839b32cc01964`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:24:14.101237 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-711db383-59b2-4b34-95f8-9d5a884e1b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:14.220204 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1235/2462] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:24:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:24:14.233059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577db037-48bb-496b-9a59-f03c814e5135 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:14.234783 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-577db037-48bb-496b-9a59-f03c814e5135 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:14.291303 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577db037-48bb-496b-9a59-f03c814e5135 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:14.295837 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfca426-fafd-4732-ad36-30071d8e2eaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:14.296824 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfca426-fafd-4732-ad36-30071d8e2eaf service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:14.297253 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfca426-fafd-4732-ad36-30071d8e2eaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:14.297927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfca426-fafd-4732-ad36-30071d8e2eaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:14.305406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:14.311033 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfca426-fafd-4732-ad36-30071d8e2eaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:14.313026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfca426-fafd-4732-ad36-30071d8e2eaf service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:14.322834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1228/2463] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:24:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:24:14.348449 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711db383-59b2-4b34-95f8-9d5a884e1b59 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:14.414960 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711db383-59b2-4b34-95f8-9d5a884e1b59 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:14.420228 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e25997-c02d-4cb3-8303-7761ccfa2563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:14.421098 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e25997-c02d-4cb3-8303-7761ccfa2563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:14.421381 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e25997-c02d-4cb3-8303-7761ccfa2563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:14.421658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e25997-c02d-4cb3-8303-7761ccfa2563 admin admin] PATH_INFO: `/v3/users/4fafbf3aec7a48e6b67839b32cc01964` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:14.422930 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e25997-c02d-4cb3-8303-7761ccfa2563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fafbf3aec7a48e6b67839b32cc01964)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:14.424994 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e25997-c02d-4cb3-8303-7761ccfa2563 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:14.475659 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-60e25997-c02d-4cb3-8303-7761ccfa2563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fafbf3aec7a48e6b67839b32cc01964'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:14.488973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-60e25997-c02d-4cb3-8303-7761ccfa2563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fafbf3aec7a48e6b67839b32cc01964'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:14.489669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-60e25997-c02d-4cb3-8303-7761ccfa2563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fafbf3aec7a48e6b67839b32cc01964'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:14.490267 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-60e25997-c02d-4cb3-8303-7761ccfa2563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fafbf3aec7a48e6b67839b32cc01964'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:14.497875 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-60e25997-c02d-4cb3-8303-7761ccfa2563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fafbf3aec7a48e6b67839b32cc01964'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:14.506687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:14.516914 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1236/2464] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:24:14 2022] DELETE /identity/v3/users/4fafbf3aec7a48e6b67839b32cc01964 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:24:14.517616 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c413758-ec3d-4891-be0c-a04867941d8b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:14.517908 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c413758-ec3d-4891-be0c-a04867941d8b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:14.518187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c413758-ec3d-4891-be0c-a04867941d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:14.535091 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9c413758-ec3d-4891-be0c-a04867941d8b None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:24:14.536358 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9c413758-ec3d-4891-be0c-a04867941d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:14.594045 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1229/2465] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:24:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:24:14.604128 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e25997-c02d-4cb3-8303-7761ccfa2563 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:14.606111 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-60e25997-c02d-4cb3-8303-7761ccfa2563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:14.663642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e25997-c02d-4cb3-8303-7761ccfa2563 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:14.668067 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117deaf-9d7a-4598-9455-51ade59755ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:14.668843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117deaf-9d7a-4598-9455-51ade59755ca service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:14.670628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117deaf-9d7a-4598-9455-51ade59755ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:14.670835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117deaf-9d7a-4598-9455-51ade59755ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:14.681607 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3117deaf-9d7a-4598-9455-51ade59755ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:14.683724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3117deaf-9d7a-4598-9455-51ade59755ca service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:14.693972 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1237/2466] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:24:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:24:15.116300 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c413758-ec3d-4891-be0c-a04867941d8b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:15.117862 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9c413758-ec3d-4891-be0c-a04867941d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:15.171530 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c413758-ec3d-4891-be0c-a04867941d8b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:15.192413 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e830b-4251-49c9-9e22-19104badd4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:15.194829 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e830b-4251-49c9-9e22-19104badd4b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:15.195439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e830b-4251-49c9-9e22-19104badd4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:15.197128 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e830b-4251-49c9-9e22-19104badd4b7 admin admin] PATH_INFO: `/v3/projects/fcddbc7dcdc64b95ae7a63348254cba9` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:15.203252 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e830b-4251-49c9-9e22-19104badd4b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcddbc7dcdc64b95ae7a63348254cba9)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:15.207382 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e830b-4251-49c9-9e22-19104badd4b7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:15.289815 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f29e830b-4251-49c9-9e22-19104badd4b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcddbc7dcdc64b95ae7a63348254cba9', 'request_id': 'req-f29e830b-4251-49c9-9e22-19104badd4b7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:15.299932 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f29e830b-4251-49c9-9e22-19104badd4b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcddbc7dcdc64b95ae7a63348254cba9', 'request_id': 'req-f29e830b-4251-49c9-9e22-19104badd4b7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:15.306458 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f29e830b-4251-49c9-9e22-19104badd4b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcddbc7dcdc64b95ae7a63348254cba9', 'request_id': 'req-f29e830b-4251-49c9-9e22-19104badd4b7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:15.320000 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f29e830b-4251-49c9-9e22-19104badd4b7 admin admin] The token cache is being invalidate because project fcddbc7dcdc64b95ae7a63348254cba9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:24:15.320220 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f29e830b-4251-49c9-9e22-19104badd4b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:15.321123 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f29e830b-4251-49c9-9e22-19104badd4b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:15.322803 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1230/2467] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:24:15 2022] DELETE /identity/v3/projects/fcddbc7dcdc64b95ae7a63348254cba9 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:24:15.327748 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062003c2-5971-4710-8e44-1d5539661fa6 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:15.328538 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062003c2-5971-4710-8e44-1d5539661fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:15.328538 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062003c2-5971-4710-8e44-1d5539661fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:15.348970 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-062003c2-5971-4710-8e44-1d5539661fa6 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:24:15.350192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-062003c2-5971-4710-8e44-1d5539661fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:15.431536 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1238/2468] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:24:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:24:15.436415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14f307-0477-40bd-86ca-d491452c73c0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:15.436905 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14f307-0477-40bd-86ca-d491452c73c0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:15.437281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14f307-0477-40bd-86ca-d491452c73c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:15.461295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1b14f307-0477-40bd-86ca-d491452c73c0 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:24:15.462639 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1b14f307-0477-40bd-86ca-d491452c73c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:15.472428 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1231/2469] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:24:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:24:15.479395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062003c2-5971-4710-8e44-1d5539661fa6 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:15.490043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062003c2-5971-4710-8e44-1d5539661fa6 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:15.495512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f9ac1-8560-4cf8-9877-ab6a4f3ff91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:15.496383 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f9ac1-8560-4cf8-9877-ab6a4f3ff91f admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:15.496473 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f9ac1-8560-4cf8-9877-ab6a4f3ff91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:15.496707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f9ac1-8560-4cf8-9877-ab6a4f3ff91f admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:15.497438 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f9ac1-8560-4cf8-9877-ab6a4f3ff91f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:24:15.497743 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f9ac1-8560-4cf8-9877-ab6a4f3ff91f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:15.499513 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f9ac1-8560-4cf8-9877-ab6a4f3ff91f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:15.512735 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1239/2470] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:24:15.516847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14f307-0477-40bd-86ca-d491452c73c0 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:15.527190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14f307-0477-40bd-86ca-d491452c73c0 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:15.532138 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38b996-ac3c-4f27-8236-9c35347b09e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:15.532811 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38b996-ac3c-4f27-8236-9c35347b09e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:15.533096 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38b996-ac3c-4f27-8236-9c35347b09e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:15.533372 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38b996-ac3c-4f27-8236-9c35347b09e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:15.534071 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38b996-ac3c-4f27-8236-9c35347b09e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:15.536101 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38b996-ac3c-4f27-8236-9c35347b09e3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:15.563935 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1232/2471] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:24:15 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:24:15.568273 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f9ac1-8560-4cf8-9877-ab6a4f3ff91f admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:15.583011 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f9ac1-8560-4cf8-9877-ab6a4f3ff91f admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:15.643893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031aeff8-4182-41c1-9593-c313395220fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:15.644641 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031aeff8-4182-41c1-9593-c313395220fd admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:15.644838 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031aeff8-4182-41c1-9593-c313395220fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:15.645112 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031aeff8-4182-41c1-9593-c313395220fd admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:15.645794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031aeff8-4182-41c1-9593-c313395220fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:15.647727 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031aeff8-4182-41c1-9593-c313395220fd admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:15.668455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1240/2472] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:24:15 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:24:15.672706 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38b996-ac3c-4f27-8236-9c35347b09e3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:15.683657 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38b996-ac3c-4f27-8236-9c35347b09e3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:15.688432 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4849a-bf3c-47fe-b363-558709c38c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:15.689101 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4849a-bf3c-47fe-b363-558709c38c50 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:15.689377 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4849a-bf3c-47fe-b363-558709c38c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:15.689647 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4849a-bf3c-47fe-b363-558709c38c50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:15.690372 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e4849a-bf3c-47fe-b363-558709c38c50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:15.692091 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e4849a-bf3c-47fe-b363-558709c38c50 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:15.703418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1233/2473] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:24:15.707619 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031aeff8-4182-41c1-9593-c313395220fd admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:15.717726 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031aeff8-4182-41c1-9593-c313395220fd admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:15.722539 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399eb94-5e09-41ff-9ad8-ea0449abd84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:15.723207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399eb94-5e09-41ff-9ad8-ea0449abd84f admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:15.723514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399eb94-5e09-41ff-9ad8-ea0449abd84f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:15.723797 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399eb94-5e09-41ff-9ad8-ea0449abd84f admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:15.724523 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399eb94-5e09-41ff-9ad8-ea0449abd84f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:15.726098 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399eb94-5e09-41ff-9ad8-ea0449abd84f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:15.735827 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1241/2474] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:24:15.740469 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4849a-bf3c-47fe-b363-558709c38c50 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:15.751499 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4849a-bf3c-47fe-b363-558709c38c50 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:15.756751 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3f819-d0ef-416d-a891-ba54b8aadafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:15.757733 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3f819-d0ef-416d-a891-ba54b8aadafc admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:15.758068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3f819-d0ef-416d-a891-ba54b8aadafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:15.758377 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3f819-d0ef-416d-a891-ba54b8aadafc admin admin] PATH_INFO: `/v3/projects/29e11f4f22b54756b307db8736e277c9/users/d96d305c6c4443f3ae4b9912c103aff4/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:15.774557 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3f819-d0ef-416d-a891-ba54b8aadafc admin admin] RBAC: Authorizing `identity:create_grant(project_id=29e11f4f22b54756b307db8736e277c9, user_id=d96d305c6c4443f3ae4b9912c103aff4, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:15.777443 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3f819-d0ef-416d-a891-ba54b8aadafc admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:15.790297 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1234/2475] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:24:15 2022] PUT /identity/v3/projects/29e11f4f22b54756b307db8736e277c9/users/d96d305c6c4443f3ae4b9912c103aff4/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:24:15.802120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047829ed-2899-48c6-83d6-bc825a4d1fa6 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:15.802406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047829ed-2899-48c6-83d6-bc825a4d1fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:15.802680 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047829ed-2899-48c6-83d6-bc825a4d1fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:15.815754 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:15.830659 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-047829ed-2899-48c6-83d6-bc825a4d1fa6 None None] MFA Rules not processed for user `d96d305c6c4443f3ae4b9912c103aff4`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:24:15.831926 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-047829ed-2899-48c6-83d6-bc825a4d1fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:15.902835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1242/2476] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:24:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:24:15.913697 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3f819-d0ef-416d-a891-ba54b8aadafc admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:15.915384 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fcd3f819-d0ef-416d-a891-ba54b8aadafc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:15.930489 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:15.977614 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3f819-d0ef-416d-a891-ba54b8aadafc admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:15.981428 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22d6e2-efbe-4006-bac6-29187c8df355 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:15.982082 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22d6e2-efbe-4006-bac6-29187c8df355 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:15.982368 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22d6e2-efbe-4006-bac6-29187c8df355 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:15.982648 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22d6e2-efbe-4006-bac6-29187c8df355 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:15.990968 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22d6e2-efbe-4006-bac6-29187c8df355 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:15.992509 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22d6e2-efbe-4006-bac6-29187c8df355 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:16.001522 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1235/2477] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:24:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:24:25.359927 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047829ed-2899-48c6-83d6-bc825a4d1fa6 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:25.361323 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-047829ed-2899-48c6-83d6-bc825a4d1fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:25.455621 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047829ed-2899-48c6-83d6-bc825a4d1fa6 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:25.461232 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895dc0a-fd64-45b7-a93d-d246353eceec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab73cf3febbf4c768cd0116306f16c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:25.462298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895dc0a-fd64-45b7-a93d-d246353eceec service placement] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:25.462298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895dc0a-fd64-45b7-a93d-d246353eceec service placement] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:25.462537 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895dc0a-fd64-45b7-a93d-d246353eceec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:25.464929 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d895dc0a-fd64-45b7-a93d-d246353eceec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:25.477810 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895dc0a-fd64-45b7-a93d-d246353eceec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:25.480182 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895dc0a-fd64-45b7-a93d-d246353eceec service placement] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:25.491042 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1243/2478] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:24:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:24:25.997446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=20 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:35.486130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=70 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:35.885559 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649695d-2535-4984-9af0-d14c71f75d99 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:35.886086 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649695d-2535-4984-9af0-d14c71f75d99 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:35.886282 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649695d-2535-4984-9af0-d14c71f75d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:35.946820 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6649695d-2535-4984-9af0-d14c71f75d99 None None] MFA Rules not processed for user `d00b8bc311234e6da47b365ac8a7183a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:24:35.948271 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6649695d-2535-4984-9af0-d14c71f75d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:36.027263 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1236/2479] 10.222.0.81 () {58 vars in 1009 bytes} [Tue Jun 7 14:24:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:24:36.076645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895dc0a-fd64-45b7-a93d-d246353eceec service placement] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:36.078113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d895dc0a-fd64-45b7-a93d-d246353eceec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:36.184055 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895dc0a-fd64-45b7-a93d-d246353eceec service placement] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:36.189601 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8479d4ef-e4d5-440c-8ef6-a06bf20b543b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da7c0f99060a44eaab743ba71f159e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:36.190255 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479d4ef-e4d5-440c-8ef6-a06bf20b543b service swift] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:36.190538 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479d4ef-e4d5-440c-8ef6-a06bf20b543b service swift] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:36.190861 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479d4ef-e4d5-440c-8ef6-a06bf20b543b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:36.203424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8479d4ef-e4d5-440c-8ef6-a06bf20b543b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:36.205035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8479d4ef-e4d5-440c-8ef6-a06bf20b543b service swift] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:36.214044 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1244/2480] 10.222.0.81 () {60 vars in 1384 bytes} [Tue Jun 7 14:24:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:24:38.430434 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649695d-2535-4984-9af0-d14c71f75d99 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:38.566033 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649695d-2535-4984-9af0-d14c71f75d99 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:38.573586 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f809d86-9c2d-419a-bb79-c6544b871b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:38.575979 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f809d86-9c2d-419a-bb79-c6544b871b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:38.576268 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f809d86-9c2d-419a-bb79-c6544b871b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:38.576548 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f809d86-9c2d-419a-bb79-c6544b871b58 admin admin] PATH_INFO: `/v3/users/d96d305c6c4443f3ae4b9912c103aff4` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:38.577927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f809d86-9c2d-419a-bb79-c6544b871b58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d96d305c6c4443f3ae4b9912c103aff4)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:38.580362 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f809d86-9c2d-419a-bb79-c6544b871b58 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:38.654162 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0f809d86-9c2d-419a-bb79-c6544b871b58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd96d305c6c4443f3ae4b9912c103aff4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:38.655034 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0f809d86-9c2d-419a-bb79-c6544b871b58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd96d305c6c4443f3ae4b9912c103aff4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:38.655899 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0f809d86-9c2d-419a-bb79-c6544b871b58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd96d305c6c4443f3ae4b9912c103aff4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:38.667068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0f809d86-9c2d-419a-bb79-c6544b871b58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd96d305c6c4443f3ae4b9912c103aff4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:38.670415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:38.692109 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0f809d86-9c2d-419a-bb79-c6544b871b58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd96d305c6c4443f3ae4b9912c103aff4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:38.710574 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1237/2481] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:24:38 2022] DELETE /identity/v3/users/d96d305c6c4443f3ae4b9912c103aff4 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:24:38.715070 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2448f5-5733-40b1-a853-fff29f16f983 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:38.715717 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2448f5-5733-40b1-a853-fff29f16f983 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:38.715820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2448f5-5733-40b1-a853-fff29f16f983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:38.755954 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-8e2448f5-5733-40b1-a853-fff29f16f983 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:24:38.757210 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8e2448f5-5733-40b1-a853-fff29f16f983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:38.869880 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1245/2482] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:24:38.888649 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f809d86-9c2d-419a-bb79-c6544b871b58 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:38.893229 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0f809d86-9c2d-419a-bb79-c6544b871b58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:38.970030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f809d86-9c2d-419a-bb79-c6544b871b58 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:38.974602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab89a3-f87c-4ced-966b-349d496e49c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:38.975401 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab89a3-f87c-4ced-966b-349d496e49c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:38.975802 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab89a3-f87c-4ced-966b-349d496e49c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:38.976183 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab89a3-f87c-4ced-966b-349d496e49c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:38.987710 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dab89a3-f87c-4ced-966b-349d496e49c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:38.989464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dab89a3-f87c-4ced-966b-349d496e49c1 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:39.022142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1238/2483] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:24:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:24:39.676033 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2448f5-5733-40b1-a853-fff29f16f983 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:39.677059 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8e2448f5-5733-40b1-a853-fff29f16f983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:39.699627 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2448f5-5733-40b1-a853-fff29f16f983 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:39.704904 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58096cd-4d14-41b7-b6f5-6ce1214c8b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:39.705774 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58096cd-4d14-41b7-b6f5-6ce1214c8b2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:39.706060 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58096cd-4d14-41b7-b6f5-6ce1214c8b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:39.706371 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58096cd-4d14-41b7-b6f5-6ce1214c8b2c admin admin] PATH_INFO: `/v3/projects/29e11f4f22b54756b307db8736e277c9` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:39.707616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58096cd-4d14-41b7-b6f5-6ce1214c8b2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=29e11f4f22b54756b307db8736e277c9)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:39.709547 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58096cd-4d14-41b7-b6f5-6ce1214c8b2c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:39.776043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c58096cd-4d14-41b7-b6f5-6ce1214c8b2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29e11f4f22b54756b307db8736e277c9', 'request_id': 'req-c58096cd-4d14-41b7-b6f5-6ce1214c8b2c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:39.788059 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c58096cd-4d14-41b7-b6f5-6ce1214c8b2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29e11f4f22b54756b307db8736e277c9', 'request_id': 'req-c58096cd-4d14-41b7-b6f5-6ce1214c8b2c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:39.795674 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c58096cd-4d14-41b7-b6f5-6ce1214c8b2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29e11f4f22b54756b307db8736e277c9', 'request_id': 'req-c58096cd-4d14-41b7-b6f5-6ce1214c8b2c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:39.800705 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:39.812663 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c58096cd-4d14-41b7-b6f5-6ce1214c8b2c admin admin] The token cache is being invalidate because project 29e11f4f22b54756b307db8736e277c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:24:39.812966 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c58096cd-4d14-41b7-b6f5-6ce1214c8b2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:39.814288 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c58096cd-4d14-41b7-b6f5-6ce1214c8b2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:39.816155 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1246/2484] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:24:39 2022] DELETE /identity/v3/projects/29e11f4f22b54756b307db8736e277c9 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:24:40.656896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab89a3-f87c-4ced-966b-349d496e49c1 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:40.665680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab89a3-f87c-4ced-966b-349d496e49c1 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:40.730635 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b00c9-6fd3-4be3-930a-378f96ecc781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:40.731507 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b00c9-6fd3-4be3-930a-378f96ecc781 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:40.732668 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b00c9-6fd3-4be3-930a-378f96ecc781 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:40.733143 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b00c9-6fd3-4be3-930a-378f96ecc781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:40.735636 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-765b00c9-6fd3-4be3-930a-378f96ecc781 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:40.827065 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b00c9-6fd3-4be3-930a-378f96ecc781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:40.828812 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b00c9-6fd3-4be3-930a-378f96ecc781 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:40.839400 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1239/2485] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:24:40 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:24:49.813785 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:DELETE=1,keystone:SELECT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:50.834779 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=83,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:55.269452 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171d156-34d1-44f6-8061-1851522bb372 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:55.269945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171d156-34d1-44f6-8061-1851522bb372 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:55.270059 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171d156-34d1-44f6-8061-1851522bb372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:55.288101 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-4171d156-34d1-44f6-8061-1851522bb372 None None] MFA Rules not processed for user `d00b8bc311234e6da47b365ac8a7183a`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:24:55.289303 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4171d156-34d1-44f6-8061-1851522bb372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:55.345793 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1247/2486] 10.222.0.81 () {58 vars in 967 bytes} [Tue Jun 7 14:24:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:24:55.359134 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b00c9-6fd3-4be3-930a-378f96ecc781 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:55.360560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-765b00c9-6fd3-4be3-930a-378f96ecc781 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:55.429590 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b00c9-6fd3-4be3-930a-378f96ecc781 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:55.434570 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c791a26-91cd-4aa3-aaac-50016cd50af8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da7c0f99060a44eaab743ba71f159e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:55.435210 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c791a26-91cd-4aa3-aaac-50016cd50af8 service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:55.435517 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c791a26-91cd-4aa3-aaac-50016cd50af8 service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:55.435788 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c791a26-91cd-4aa3-aaac-50016cd50af8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:55.443640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c791a26-91cd-4aa3-aaac-50016cd50af8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:55.445205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c791a26-91cd-4aa3-aaac-50016cd50af8 service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:55.453437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1240/2487] 10.222.0.81 () {60 vars in 1384 bytes} [Tue Jun 7 14:24:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:24:59.472055 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171d156-34d1-44f6-8061-1851522bb372 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:59.473536 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4171d156-34d1-44f6-8061-1851522bb372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:59.541069 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171d156-34d1-44f6-8061-1851522bb372 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:59.545604 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71518d87-7631-410f-99c7-a4f57cba492b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:59.546493 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71518d87-7631-410f-99c7-a4f57cba492b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:59.546780 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71518d87-7631-410f-99c7-a4f57cba492b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:59.547054 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71518d87-7631-410f-99c7-a4f57cba492b admin admin] PATH_INFO: `/v3/users/becdcd1df4a641e6a5bda05220f7aa07` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:59.548320 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71518d87-7631-410f-99c7-a4f57cba492b admin admin] RBAC: Authorizing `identity:delete_user(user_id=becdcd1df4a641e6a5bda05220f7aa07)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:59.550231 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71518d87-7631-410f-99c7-a4f57cba492b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:59.594680 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-71518d87-7631-410f-99c7-a4f57cba492b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'becdcd1df4a641e6a5bda05220f7aa07'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:59.607442 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-71518d87-7631-410f-99c7-a4f57cba492b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'becdcd1df4a641e6a5bda05220f7aa07'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:59.608113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-71518d87-7631-410f-99c7-a4f57cba492b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'becdcd1df4a641e6a5bda05220f7aa07'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:59.608716 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-71518d87-7631-410f-99c7-a4f57cba492b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'becdcd1df4a641e6a5bda05220f7aa07'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:59.615481 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-71518d87-7631-410f-99c7-a4f57cba492b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'becdcd1df4a641e6a5bda05220f7aa07'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:59.629131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1248/2488] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:24:59 2022] DELETE /identity/v3/users/becdcd1df4a641e6a5bda05220f7aa07 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:24:59.802517 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c791a26-91cd-4aa3-aaac-50016cd50af8 service swift] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:59.819827 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c791a26-91cd-4aa3-aaac-50016cd50af8 service swift] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:59.876315 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73c365-87eb-4dcb-8b52-ac6b2d77ca90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:59.877174 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73c365-87eb-4dcb-8b52-ac6b2d77ca90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:59.877456 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73c365-87eb-4dcb-8b52-ac6b2d77ca90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:59.877732 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73c365-87eb-4dcb-8b52-ac6b2d77ca90 admin admin] PATH_INFO: `/v3/projects/ecb2b7cbdaf9495480fc881e823c1913` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:59.878910 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e73c365-87eb-4dcb-8b52-ac6b2d77ca90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecb2b7cbdaf9495480fc881e823c1913)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:59.880846 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e73c365-87eb-4dcb-8b52-ac6b2d77ca90 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:59.926747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0e73c365-87eb-4dcb-8b52-ac6b2d77ca90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecb2b7cbdaf9495480fc881e823c1913', 'request_id': 'req-0e73c365-87eb-4dcb-8b52-ac6b2d77ca90'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:59.933744 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0e73c365-87eb-4dcb-8b52-ac6b2d77ca90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecb2b7cbdaf9495480fc881e823c1913', 'request_id': 'req-0e73c365-87eb-4dcb-8b52-ac6b2d77ca90'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:59.939249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0e73c365-87eb-4dcb-8b52-ac6b2d77ca90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecb2b7cbdaf9495480fc881e823c1913', 'request_id': 'req-0e73c365-87eb-4dcb-8b52-ac6b2d77ca90'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:59.950678 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0e73c365-87eb-4dcb-8b52-ac6b2d77ca90 admin admin] The token cache is being invalidate because project ecb2b7cbdaf9495480fc881e823c1913 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:24:59.951165 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0e73c365-87eb-4dcb-8b52-ac6b2d77ca90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:59.951831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0e73c365-87eb-4dcb-8b52-ac6b2d77ca90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:24:59.953481 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1241/2489] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:24:59 2022] DELETE /identity/v3/projects/ecb2b7cbdaf9495480fc881e823c1913 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:24:59.958915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbcf0b-8a16-4182-89d5-e88c57aba3a0 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:59.959208 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbcf0b-8a16-4182-89d5-e88c57aba3a0 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:59.959521 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbcf0b-8a16-4182-89d5-e88c57aba3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:59.978435 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-43fbcf0b-8a16-4182-89d5-e88c57aba3a0 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:24:59.979754 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-43fbcf0b-8a16-4182-89d5-e88c57aba3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:59.999988 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:25:00.042692 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1249/2490] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:24:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:25:00.107428 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a028e-d372-400a-a130-e232876cfc3a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:00.107660 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a028e-d372-400a-a130-e232876cfc3a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:00.107891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a028e-d372-400a-a130-e232876cfc3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:00.126858 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1a1a028e-d372-400a-a130-e232876cfc3a None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:25:00.128103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1a1a028e-d372-400a-a130-e232876cfc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:25:00.136070 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1242/2491] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:25:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:25:00.140325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbcf0b-8a16-4182-89d5-e88c57aba3a0 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:25:00.150417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbcf0b-8a16-4182-89d5-e88c57aba3a0 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:25:00.155022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09266e8e-295c-445e-82b4-25f3d6e7d39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:25:00.155715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09266e8e-295c-445e-82b4-25f3d6e7d39f admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:00.155999 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09266e8e-295c-445e-82b4-25f3d6e7d39f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:00.156276 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09266e8e-295c-445e-82b4-25f3d6e7d39f admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:00.157005 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09266e8e-295c-445e-82b4-25f3d6e7d39f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:25:00.157310 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09266e8e-295c-445e-82b4-25f3d6e7d39f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:25:00.158893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09266e8e-295c-445e-82b4-25f3d6e7d39f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:25:00.172094 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1250/2492] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:25:00.176500 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a028e-d372-400a-a130-e232876cfc3a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:25:00.185871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a028e-d372-400a-a130-e232876cfc3a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:25:00.190434 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d942f4f-c888-4775-9144-8cf344ccd3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:25:00.191078 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d942f4f-c888-4775-9144-8cf344ccd3fa admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:00.191389 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d942f4f-c888-4775-9144-8cf344ccd3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:00.191665 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d942f4f-c888-4775-9144-8cf344ccd3fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:00.192350 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d942f4f-c888-4775-9144-8cf344ccd3fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:25:00.194224 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d942f4f-c888-4775-9144-8cf344ccd3fa admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:25:00.205025 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:25:00.222131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1243/2493] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:25:00 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:25:00.226317 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09266e8e-295c-445e-82b4-25f3d6e7d39f admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:25:00.236084 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09266e8e-295c-445e-82b4-25f3d6e7d39f admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:25:00.291443 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e7512-2fad-4456-89ff-0e847467f614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:25:00.292107 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e7512-2fad-4456-89ff-0e847467f614 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:00.292387 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e7512-2fad-4456-89ff-0e847467f614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:00.292658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e7512-2fad-4456-89ff-0e847467f614 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:00.293350 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e7512-2fad-4456-89ff-0e847467f614 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:25:00.295282 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e7512-2fad-4456-89ff-0e847467f614 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:25:00.315553 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1251/2494] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:25:00 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:25:00.319579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d942f4f-c888-4775-9144-8cf344ccd3fa admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:25:00.330305 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d942f4f-c888-4775-9144-8cf344ccd3fa admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:25:00.334832 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ead838-d6af-4a17-8a56-ce41eb2238da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:25:00.335521 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ead838-d6af-4a17-8a56-ce41eb2238da admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:00.335806 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ead838-d6af-4a17-8a56-ce41eb2238da admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:00.336083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ead838-d6af-4a17-8a56-ce41eb2238da admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:00.337521 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ead838-d6af-4a17-8a56-ce41eb2238da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:25:00.339123 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ead838-d6af-4a17-8a56-ce41eb2238da admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:25:00.349590 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1244/2495] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:25:00.353752 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e7512-2fad-4456-89ff-0e847467f614 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:25:00.363578 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e7512-2fad-4456-89ff-0e847467f614 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:25:00.368041 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e02b0f-8c14-4647-854a-199f72712a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:25:00.368968 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e02b0f-8c14-4647-854a-199f72712a0c admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:00.369250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e02b0f-8c14-4647-854a-199f72712a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:00.369528 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e02b0f-8c14-4647-854a-199f72712a0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:00.370249 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e02b0f-8c14-4647-854a-199f72712a0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:25:00.371849 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e02b0f-8c14-4647-854a-199f72712a0c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:25:00.381517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1252/2496] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:25:00.386029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ead838-d6af-4a17-8a56-ce41eb2238da admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:25:00.395681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ead838-d6af-4a17-8a56-ce41eb2238da admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:25:00.400926 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95dea67-cc6b-4b08-90b5-7cd524c7b700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:25:00.401792 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dea67-cc6b-4b08-90b5-7cd524c7b700 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:00.402073 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dea67-cc6b-4b08-90b5-7cd524c7b700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:00.402347 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dea67-cc6b-4b08-90b5-7cd524c7b700 admin admin] PATH_INFO: `/v3/projects/5746dfb053c64433a0c5d11d4ff70462/users/2ae6a34b7e8a46aca75cca60d41be1c9/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:00.416947 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95dea67-cc6b-4b08-90b5-7cd524c7b700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5746dfb053c64433a0c5d11d4ff70462, user_id=2ae6a34b7e8a46aca75cca60d41be1c9, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:25:00.419718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95dea67-cc6b-4b08-90b5-7cd524c7b700 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:25:00.432422 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1245/2497] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:25:00 2022] PUT /identity/v3/projects/5746dfb053c64433a0c5d11d4ff70462/users/2ae6a34b7e8a46aca75cca60d41be1c9/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:25:00.442090 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc22d63-5b64-4b07-ba71-68fc2b56ec57 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:00.442382 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc22d63-5b64-4b07-ba71-68fc2b56ec57 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:00.442660 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc22d63-5b64-4b07-ba71-68fc2b56ec57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:00.470137 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-bfc22d63-5b64-4b07-ba71-68fc2b56ec57 None None] MFA Rules not processed for user `2ae6a34b7e8a46aca75cca60d41be1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:25:00.471687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-bfc22d63-5b64-4b07-ba71-68fc2b56ec57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:25:00.534588 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1253/2498] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:25:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:25:00.545264 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95dea67-cc6b-4b08-90b5-7cd524c7b700 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:25:00.546984 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f95dea67-cc6b-4b08-90b5-7cd524c7b700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:25:00.604312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95dea67-cc6b-4b08-90b5-7cd524c7b700 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:25:00.607930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59348bd6-5119-4d6b-b567-307c89aa0682 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:25:00.608572 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59348bd6-5119-4d6b-b567-307c89aa0682 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:00.608855 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59348bd6-5119-4d6b-b567-307c89aa0682 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:00.609132 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59348bd6-5119-4d6b-b567-307c89aa0682 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:00.617760 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59348bd6-5119-4d6b-b567-307c89aa0682 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:25:00.619935 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59348bd6-5119-4d6b-b567-307c89aa0682 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:25:00.628511 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1246/2499] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:25:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:25:10.531821 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:25:10.624663 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=44,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:25:12.516204 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc22d63-5b64-4b07-ba71-68fc2b56ec57 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:25:12.582518 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc22d63-5b64-4b07-ba71-68fc2b56ec57 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:25:12.588100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e810bab9-5543-40c2-b348-4c200da0ed81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:25:12.588985 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810bab9-5543-40c2-b348-4c200da0ed81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:12.589345 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810bab9-5543-40c2-b348-4c200da0ed81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:12.589662 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810bab9-5543-40c2-b348-4c200da0ed81 admin admin] PATH_INFO: `/v3/users/2ae6a34b7e8a46aca75cca60d41be1c9` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:12.590940 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e810bab9-5543-40c2-b348-4c200da0ed81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ae6a34b7e8a46aca75cca60d41be1c9)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:25:12.593092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e810bab9-5543-40c2-b348-4c200da0ed81 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:25:12.647568 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e810bab9-5543-40c2-b348-4c200da0ed81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ae6a34b7e8a46aca75cca60d41be1c9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:25:12.660155 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e810bab9-5543-40c2-b348-4c200da0ed81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ae6a34b7e8a46aca75cca60d41be1c9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:25:12.660838 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e810bab9-5543-40c2-b348-4c200da0ed81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ae6a34b7e8a46aca75cca60d41be1c9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:25:12.661441 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e810bab9-5543-40c2-b348-4c200da0ed81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ae6a34b7e8a46aca75cca60d41be1c9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:25:12.669128 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e810bab9-5543-40c2-b348-4c200da0ed81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ae6a34b7e8a46aca75cca60d41be1c9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:25:12.683545 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1254/2500] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:25:12 2022] DELETE /identity/v3/users/2ae6a34b7e8a46aca75cca60d41be1c9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:25:12.687963 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06645680-9820-42c1-af0d-3ca38e1e5f0b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:12.688455 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06645680-9820-42c1-af0d-3ca38e1e5f0b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:12.688559 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06645680-9820-42c1-af0d-3ca38e1e5f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:12.705920 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-06645680-9820-42c1-af0d-3ca38e1e5f0b None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:25:12.707101 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-06645680-9820-42c1-af0d-3ca38e1e5f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:25:12.765722 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1247/2501] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:25:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:25:12.775420 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e810bab9-5543-40c2-b348-4c200da0ed81 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:25:12.776818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e810bab9-5543-40c2-b348-4c200da0ed81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:25:12.832726 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e810bab9-5543-40c2-b348-4c200da0ed81 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:25:12.836819 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a2b6b6-5fd4-4e33-a3e2-357c2370bc16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:25:12.837709 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2b6b6-5fd4-4e33-a3e2-357c2370bc16 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:12.837987 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2b6b6-5fd4-4e33-a3e2-357c2370bc16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:12.838263 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2b6b6-5fd4-4e33-a3e2-357c2370bc16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:12.847545 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a2b6b6-5fd4-4e33-a3e2-357c2370bc16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:25:12.848949 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a2b6b6-5fd4-4e33-a3e2-357c2370bc16 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:25:12.859082 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1255/2502] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:25:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:25:13.129414 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06645680-9820-42c1-af0d-3ca38e1e5f0b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:25:13.131189 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-06645680-9820-42c1-af0d-3ca38e1e5f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:25:13.143273 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06645680-9820-42c1-af0d-3ca38e1e5f0b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:25:13.147922 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f30ae-ed29-4f94-90c3-52e826c5d12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:25:13.148904 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f30ae-ed29-4f94-90c3-52e826c5d12d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:13.149246 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f30ae-ed29-4f94-90c3-52e826c5d12d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:13.149701 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f30ae-ed29-4f94-90c3-52e826c5d12d admin admin] PATH_INFO: `/v3/projects/5746dfb053c64433a0c5d11d4ff70462` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:13.150996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f30ae-ed29-4f94-90c3-52e826c5d12d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5746dfb053c64433a0c5d11d4ff70462)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:25:13.157721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f30ae-ed29-4f94-90c3-52e826c5d12d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:25:13.205436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-884f30ae-ed29-4f94-90c3-52e826c5d12d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5746dfb053c64433a0c5d11d4ff70462', 'request_id': 'req-884f30ae-ed29-4f94-90c3-52e826c5d12d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:25:13.212903 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-884f30ae-ed29-4f94-90c3-52e826c5d12d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5746dfb053c64433a0c5d11d4ff70462', 'request_id': 'req-884f30ae-ed29-4f94-90c3-52e826c5d12d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:25:13.218500 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-884f30ae-ed29-4f94-90c3-52e826c5d12d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5746dfb053c64433a0c5d11d4ff70462', 'request_id': 'req-884f30ae-ed29-4f94-90c3-52e826c5d12d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:25:13.230675 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-884f30ae-ed29-4f94-90c3-52e826c5d12d admin admin] The token cache is being invalidate because project 5746dfb053c64433a0c5d11d4ff70462 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:25:13.230962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-884f30ae-ed29-4f94-90c3-52e826c5d12d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:25:13.231662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-884f30ae-ed29-4f94-90c3-52e826c5d12d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:25:13.233231 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1248/2503] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:25:13 2022] DELETE /identity/v3/projects/5746dfb053c64433a0c5d11d4ff70462 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:25:13.238193 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013adb8f-3e33-4b1e-bcab-a66d42c4b226 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:13.238490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013adb8f-3e33-4b1e-bcab-a66d42c4b226 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:13.238771 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013adb8f-3e33-4b1e-bcab-a66d42c4b226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:13.257041 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-013adb8f-3e33-4b1e-bcab-a66d42c4b226 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:25:13.258242 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-013adb8f-3e33-4b1e-bcab-a66d42c4b226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:25:13.275649 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:25:13.319364 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1256/2504] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:25:13.328554 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd166611-cafa-43f9-8698-3c38ea712ae2 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:13.328838 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd166611-cafa-43f9-8698-3c38ea712ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:13.329115 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd166611-cafa-43f9-8698-3c38ea712ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:13.350515 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-bd166611-cafa-43f9-8698-3c38ea712ae2 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:25:13.351741 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bd166611-cafa-43f9-8698-3c38ea712ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:25:13.360332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1249/2505] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:25:13.364776 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013adb8f-3e33-4b1e-bcab-a66d42c4b226 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:25:13.374612 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013adb8f-3e33-4b1e-bcab-a66d42c4b226 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:25:13.379341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd0427-762b-47ad-adce-8f773b984fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:25:13.379995 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd0427-762b-47ad-adce-8f773b984fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:13.380276 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd0427-762b-47ad-adce-8f773b984fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:13.380550 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd0427-762b-47ad-adce-8f773b984fc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:13.381272 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd0427-762b-47ad-adce-8f773b984fc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:25:13.381575 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd0427-762b-47ad-adce-8f773b984fc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:25:13.383281 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd0427-762b-47ad-adce-8f773b984fc0 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:25:13.394557 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1257/2506] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:25:13.399631 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd166611-cafa-43f9-8698-3c38ea712ae2 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:25:13.408901 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd166611-cafa-43f9-8698-3c38ea712ae2 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:25:13.413517 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39592ece-92a3-4f7a-9dae-5b565fb6e206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:25:13.414170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39592ece-92a3-4f7a-9dae-5b565fb6e206 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:13.415021 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39592ece-92a3-4f7a-9dae-5b565fb6e206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:13.415344 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39592ece-92a3-4f7a-9dae-5b565fb6e206 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:13.416096 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39592ece-92a3-4f7a-9dae-5b565fb6e206 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:25:13.418016 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39592ece-92a3-4f7a-9dae-5b565fb6e206 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:25:13.442343 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1250/2507] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:25:13 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:25:13.446555 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd0427-762b-47ad-adce-8f773b984fc0 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:25:13.460446 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd0427-762b-47ad-adce-8f773b984fc0 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:25:13.518745 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ef99c-8cbc-4075-9934-064c023f5dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:25:13.519447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ef99c-8cbc-4075-9934-064c023f5dda admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:13.519729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ef99c-8cbc-4075-9934-064c023f5dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:13.520005 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ef99c-8cbc-4075-9934-064c023f5dda admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:13.520720 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ef99c-8cbc-4075-9934-064c023f5dda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:25:13.522792 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ef99c-8cbc-4075-9934-064c023f5dda admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:25:13.541792 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1258/2508] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:25:13 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:25:13.545879 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39592ece-92a3-4f7a-9dae-5b565fb6e206 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:25:13.556952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39592ece-92a3-4f7a-9dae-5b565fb6e206 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:25:13.561536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ecc280-c79f-433e-a7d5-ced9dc257513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:25:13.562196 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecc280-c79f-433e-a7d5-ced9dc257513 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:13.562478 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecc280-c79f-433e-a7d5-ced9dc257513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:13.562755 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecc280-c79f-433e-a7d5-ced9dc257513 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:13.563522 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ecc280-c79f-433e-a7d5-ced9dc257513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:25:13.565140 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ecc280-c79f-433e-a7d5-ced9dc257513 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:25:13.576107 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1251/2509] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:25:13.579891 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ef99c-8cbc-4075-9934-064c023f5dda admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:25:13.589887 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ef99c-8cbc-4075-9934-064c023f5dda admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:25:13.594647 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ddf847-0159-4dc8-875e-84a3b09137e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:25:13.595367 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ddf847-0159-4dc8-875e-84a3b09137e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:13.595653 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ddf847-0159-4dc8-875e-84a3b09137e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:13.596194 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ddf847-0159-4dc8-875e-84a3b09137e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:13.596923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ddf847-0159-4dc8-875e-84a3b09137e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:25:13.598554 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ddf847-0159-4dc8-875e-84a3b09137e3 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:25:13.609540 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1259/2510] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:25:13.613536 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ecc280-c79f-433e-a7d5-ced9dc257513 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:25:13.624076 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ecc280-c79f-433e-a7d5-ced9dc257513 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:25:13.628709 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b54d58-5251-4e35-9e49-51ffbeac9020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:25:13.629743 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b54d58-5251-4e35-9e49-51ffbeac9020 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:13.630024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b54d58-5251-4e35-9e49-51ffbeac9020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:13.630320 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b54d58-5251-4e35-9e49-51ffbeac9020 admin admin] PATH_INFO: `/v3/projects/1c7b98807cae4a75968ed946de9c92bb/users/9bb946aacaf145d5b718f82e28c999d9/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:13.645912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b54d58-5251-4e35-9e49-51ffbeac9020 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c7b98807cae4a75968ed946de9c92bb, user_id=9bb946aacaf145d5b718f82e28c999d9, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:25:13.648606 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b54d58-5251-4e35-9e49-51ffbeac9020 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:25:13.662704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1252/2511] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:25:13 2022] PUT /identity/v3/projects/1c7b98807cae4a75968ed946de9c92bb/users/9bb946aacaf145d5b718f82e28c999d9/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:25:13.667671 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9658b8-926d-4aa0-b0ed-27f913d9c475 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:13.667960 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9658b8-926d-4aa0-b0ed-27f913d9c475 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:13.668237 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9658b8-926d-4aa0-b0ed-27f913d9c475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:13.687115 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-db9658b8-926d-4aa0-b0ed-27f913d9c475 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:25:13.688365 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-db9658b8-926d-4aa0-b0ed-27f913d9c475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:25:13.746972 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1260/2512] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:25:13.756505 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b54d58-5251-4e35-9e49-51ffbeac9020 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:25:13.758324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-19b54d58-5251-4e35-9e49-51ffbeac9020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:25:13.779884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:25:13.816927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b54d58-5251-4e35-9e49-51ffbeac9020 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:25:13.820694 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d066b-7950-4fd6-bf01-1b979a4317b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:25:13.821515 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d066b-7950-4fd6-bf01-1b979a4317b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:13.821802 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d066b-7950-4fd6-bf01-1b979a4317b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:13.822128 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d066b-7950-4fd6-bf01-1b979a4317b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:13.831295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d066b-7950-4fd6-bf01-1b979a4317b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:25:13.832711 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d066b-7950-4fd6-bf01-1b979a4317b5 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:25:13.842024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1253/2513] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:25:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:25:15.742175 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dc494-7686-4574-8fed-5f85364d9127 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:15.743456 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dc494-7686-4574-8fed-5f85364d9127 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:15.744518 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dc494-7686-4574-8fed-5f85364d9127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:15.763255 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-355dc494-7686-4574-8fed-5f85364d9127 None None] MFA Rules not processed for user `79357bc4bc0d4b8d9dd5b07a25510f3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:25:15.764466 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-355dc494-7686-4574-8fed-5f85364d9127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:25:15.789746 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:25:15.828072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1261/2514] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:25:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:25:18.791342 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee36791-067c-493c-bed4-ac0fd0ef5382 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:18.792206 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee36791-067c-493c-bed4-ac0fd0ef5382 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:18.792690 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee36791-067c-493c-bed4-ac0fd0ef5382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:18.824958 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3ee36791-067c-493c-bed4-ac0fd0ef5382 None None] MFA Rules not processed for user `9bb946aacaf145d5b718f82e28c999d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:25:18.827504 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3ee36791-067c-493c-bed4-ac0fd0ef5382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:25:18.933395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1254/2515] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:25:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:25:18.935102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dc494-7686-4574-8fed-5f85364d9127 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:25:18.952793 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dc494-7686-4574-8fed-5f85364d9127 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:25:18.959871 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86357972-dcc3-4837-9efb-dd8645f2b287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:25:18.960899 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86357972-dcc3-4837-9efb-dd8645f2b287 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:18.961402 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86357972-dcc3-4837-9efb-dd8645f2b287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:18.961808 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86357972-dcc3-4837-9efb-dd8645f2b287 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:18.962659 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86357972-dcc3-4837-9efb-dd8645f2b287 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:25:18.965084 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86357972-dcc3-4837-9efb-dd8645f2b287 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:25:18.994245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1262/2516] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:25:18 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:25:18.998291 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee36791-067c-493c-bed4-ac0fd0ef5382 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:25:19.013036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee36791-067c-493c-bed4-ac0fd0ef5382 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:25:19.075068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9a9e0-cc1a-420b-9cf0-0400489fc7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:25:19.075817 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9a9e0-cc1a-420b-9cf0-0400489fc7a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:19.076150 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9a9e0-cc1a-420b-9cf0-0400489fc7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:19.076498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9a9e0-cc1a-420b-9cf0-0400489fc7a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:19.077196 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9a9e0-cc1a-420b-9cf0-0400489fc7a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:25:19.079145 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9a9e0-cc1a-420b-9cf0-0400489fc7a6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:25:19.104589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1255/2517] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:25:18 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:25:19.105868 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86357972-dcc3-4837-9efb-dd8645f2b287 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:25:19.118340 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86357972-dcc3-4837-9efb-dd8645f2b287 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:25:19.123168 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9584b1f-9ae5-4d39-876c-212b94aef06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:25:19.124214 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9584b1f-9ae5-4d39-876c-212b94aef06e admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:19.124214 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9584b1f-9ae5-4d39-876c-212b94aef06e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:19.124450 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9584b1f-9ae5-4d39-876c-212b94aef06e admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:19.125167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9584b1f-9ae5-4d39-876c-212b94aef06e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:25:19.126775 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9584b1f-9ae5-4d39-876c-212b94aef06e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:25:19.142985 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1263/2518] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:25:19.143577 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9a9e0-cc1a-420b-9cf0-0400489fc7a6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:25:19.160260 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9a9e0-cc1a-420b-9cf0-0400489fc7a6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:25:19.171379 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02990a9-4d73-4afd-ba99-d24bc80530f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:25:19.172314 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02990a9-4d73-4afd-ba99-d24bc80530f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:19.172594 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02990a9-4d73-4afd-ba99-d24bc80530f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:19.172900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02990a9-4d73-4afd-ba99-d24bc80530f7 admin admin] PATH_INFO: `/v3/projects/2ce5417c58dd4bdd8598dd8dd2667a7d/users/3b21bef5efa8422989c4426cc3594144/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:19.192218 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02990a9-4d73-4afd-ba99-d24bc80530f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ce5417c58dd4bdd8598dd8dd2667a7d, user_id=3b21bef5efa8422989c4426cc3594144, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:25:19.194897 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02990a9-4d73-4afd-ba99-d24bc80530f7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:25:19.209095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1256/2519] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:25:19 2022] PUT /identity/v3/projects/2ce5417c58dd4bdd8598dd8dd2667a7d/users/3b21bef5efa8422989c4426cc3594144/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:25:19.213201 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9584b1f-9ae5-4d39-876c-212b94aef06e admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:25:19.223399 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9584b1f-9ae5-4d39-876c-212b94aef06e admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:25:19.291875 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5508d81-e065-40dd-9173-48feafbc2505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:25:19.292571 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5508d81-e065-40dd-9173-48feafbc2505 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:19.292860 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5508d81-e065-40dd-9173-48feafbc2505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:19.293143 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5508d81-e065-40dd-9173-48feafbc2505 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:19.293872 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5508d81-e065-40dd-9173-48feafbc2505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:25:19.295503 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5508d81-e065-40dd-9173-48feafbc2505 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:25:19.307837 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1264/2520] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:25:19.311720 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02990a9-4d73-4afd-ba99-d24bc80530f7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:25:19.323033 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02990a9-4d73-4afd-ba99-d24bc80530f7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:25:19.331932 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e483ebe-f445-46b2-8301-bcd4086cb234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:25:19.333176 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e483ebe-f445-46b2-8301-bcd4086cb234 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:19.333176 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e483ebe-f445-46b2-8301-bcd4086cb234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:19.333395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e483ebe-f445-46b2-8301-bcd4086cb234 admin admin] PATH_INFO: `/v3/projects/2ce5417c58dd4bdd8598dd8dd2667a7d/users/3b21bef5efa8422989c4426cc3594144/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:19.335423 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e483ebe-f445-46b2-8301-bcd4086cb234 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ce5417c58dd4bdd8598dd8dd2667a7d, user_id=3b21bef5efa8422989c4426cc3594144, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:25:19.338760 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e483ebe-f445-46b2-8301-bcd4086cb234 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:25:19.349924 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1257/2521] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:25:19 2022] PUT /identity/v3/projects/2ce5417c58dd4bdd8598dd8dd2667a7d/users/3b21bef5efa8422989c4426cc3594144/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:25:24.746676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854966d3-c898-43ee-86d1-8c7c43c8c63c None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:24.747511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854966d3-c898-43ee-86d1-8c7c43c8c63c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:24.748081 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854966d3-c898-43ee-86d1-8c7c43c8c63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:24.785305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-854966d3-c898-43ee-86d1-8c7c43c8c63c None None] MFA Rules not processed for user `3b21bef5efa8422989c4426cc3594144`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:25:24.786695 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-854966d3-c898-43ee-86d1-8c7c43c8c63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:25:24.887370 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1265/2522] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:25:24 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:25:24.901978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e483ebe-f445-46b2-8301-bcd4086cb234 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:25:24.904924 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6e483ebe-f445-46b2-8301-bcd4086cb234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:25:24.909817 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:25:25.005113 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e483ebe-f445-46b2-8301-bcd4086cb234 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:25:25.010873 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc483b-ac1b-4744-bed3-7fb512e8b2c6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6180393da0f495db964f0b2588d8b90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:25:25.012236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc483b-ac1b-4744-bed3-7fb512e8b2c6 service glance] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:25.012708 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc483b-ac1b-4744-bed3-7fb512e8b2c6 service glance] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:25.013156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc483b-ac1b-4744-bed3-7fb512e8b2c6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:25.079700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc483b-ac1b-4744-bed3-7fb512e8b2c6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:25:25.081348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc483b-ac1b-4744-bed3-7fb512e8b2c6 service glance] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:25:25.093036 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1258/2523] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:25:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:25:25.220839 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365f16b-f3ee-45c0-948d-7e162e12c83e None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:25.221132 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365f16b-f3ee-45c0-948d-7e162e12c83e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:25.221412 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365f16b-f3ee-45c0-948d-7e162e12c83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:25.240425 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-c365f16b-f3ee-45c0-948d-7e162e12c83e None None] MFA Rules not processed for user `d00b8bc311234e6da47b365ac8a7183a`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:25:25.241854 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c365f16b-f3ee-45c0-948d-7e162e12c83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:25:25.263433 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:25:25.309386 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1266/2524] 10.222.0.81 () {58 vars in 1009 bytes} [Tue Jun 7 14:25:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:25:25.323517 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc483b-ac1b-4744-bed3-7fb512e8b2c6 service glance] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:25:25.325392 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c3dc483b-ac1b-4744-bed3-7fb512e8b2c6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:25:25.401291 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc483b-ac1b-4744-bed3-7fb512e8b2c6 service glance] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:25:25.406415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129e1e1-37c0-4cd2-b594-52bf2ec26bcc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da7c0f99060a44eaab743ba71f159e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:25:25.407199 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129e1e1-37c0-4cd2-b594-52bf2ec26bcc service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:25.407411 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129e1e1-37c0-4cd2-b594-52bf2ec26bcc service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:25.407649 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129e1e1-37c0-4cd2-b594-52bf2ec26bcc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:25.416136 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129e1e1-37c0-4cd2-b594-52bf2ec26bcc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:25:25.417726 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129e1e1-37c0-4cd2-b594-52bf2ec26bcc service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:25:25.426156 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1259/2525] 10.222.0.81 () {60 vars in 1384 bytes} [Tue Jun 7 14:25:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:25:34.435003 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c58e2-63e9-489f-8030-e15de2eb3407 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:34.435563 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c58e2-63e9-489f-8030-e15de2eb3407 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:34.435667 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c58e2-63e9-489f-8030-e15de2eb3407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:34.452856 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-514c58e2-63e9-489f-8030-e15de2eb3407 None None] MFA Rules not processed for user `9bb946aacaf145d5b718f82e28c999d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:25:34.454096 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-514c58e2-63e9-489f-8030-e15de2eb3407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:25:34.461644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1267/2526] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:25:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:25:35.422757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=87 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:25:38.228490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08459419-ac9b-4b1f-9002-14ffd6247d17 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:38.229301 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08459419-ac9b-4b1f-9002-14ffd6247d17 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:38.229301 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08459419-ac9b-4b1f-9002-14ffd6247d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:38.265850 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-08459419-ac9b-4b1f-9002-14ffd6247d17 None None] MFA Rules not processed for user `d00b8bc311234e6da47b365ac8a7183a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:25:38.267328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-08459419-ac9b-4b1f-9002-14ffd6247d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:25:38.277323 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1260/2527] 10.222.0.81 () {58 vars in 1009 bytes} [Tue Jun 7 14:25:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:25:38.306973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c58e2-63e9-489f-8030-e15de2eb3407 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:25:38.320135 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c58e2-63e9-489f-8030-e15de2eb3407 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:25:38.326414 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885f54e-771b-4940-b201-b228922c61a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da7c0f99060a44eaab743ba71f159e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:25:38.327081 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885f54e-771b-4940-b201-b228922c61a0 service swift] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:25:38.327401 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885f54e-771b-4940-b201-b228922c61a0 service swift] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:25:38.327687 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885f54e-771b-4940-b201-b228922c61a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:25:38.339000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3885f54e-771b-4940-b201-b228922c61a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:25:38.341075 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3885f54e-771b-4940-b201-b228922c61a0 service swift] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:25:38.353877 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1268/2528] 10.222.0.81 () {60 vars in 1384 bytes} [Tue Jun 7 14:25:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:25:48.259460 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:25:48.350020 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=24 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:26:10.294622 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08459419-ac9b-4b1f-9002-14ffd6247d17 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:26:10.296155 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-08459419-ac9b-4b1f-9002-14ffd6247d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:26:10.353187 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08459419-ac9b-4b1f-9002-14ffd6247d17 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:26:10.356718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96b75b-ddde-4991-9d3a-9daff9f8f196 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:26:10.357357 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96b75b-ddde-4991-9d3a-9daff9f8f196 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:10.357635 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96b75b-ddde-4991-9d3a-9daff9f8f196 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:10.357918 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96b75b-ddde-4991-9d3a-9daff9f8f196 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:10.367711 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96b75b-ddde-4991-9d3a-9daff9f8f196 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:26:10.369312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96b75b-ddde-4991-9d3a-9daff9f8f196 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:26:10.379002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1261/2529] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:26:10 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:26:20.374598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=26 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:26:20.904441 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885f54e-771b-4940-b201-b228922c61a0 service swift] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:26:20.913796 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885f54e-771b-4940-b201-b228922c61a0 service swift] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:26:20.917950 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1f534-2d69-4248-baa1-a4e2b45f62d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:26:20.918584 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1f534-2d69-4248-baa1-a4e2b45f62d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:20.918883 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1f534-2d69-4248-baa1-a4e2b45f62d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:20.919146 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1f534-2d69-4248-baa1-a4e2b45f62d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:20.921097 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4be1f534-2d69-4248-baa1-a4e2b45f62d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:26:21.003540 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1f534-2d69-4248-baa1-a4e2b45f62d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:26:21.004961 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1f534-2d69-4248-baa1-a4e2b45f62d8 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:26:21.027669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1269/2530] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:26:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:26:22.194103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96b75b-ddde-4991-9d3a-9daff9f8f196 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:26:22.203173 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96b75b-ddde-4991-9d3a-9daff9f8f196 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:26:22.206765 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba0c5d-00dd-427a-80f0-acda6772227c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:26:22.207422 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba0c5d-00dd-427a-80f0-acda6772227c service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:22.207704 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba0c5d-00dd-427a-80f0-acda6772227c service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:22.207978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba0c5d-00dd-427a-80f0-acda6772227c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:22.209778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a8ba0c5d-00dd-427a-80f0-acda6772227c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:26:22.224085 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba0c5d-00dd-427a-80f0-acda6772227c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:26:22.225493 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba0c5d-00dd-427a-80f0-acda6772227c service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:26:22.236956 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1262/2531] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:26:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:26:26.611549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1f534-2d69-4248-baa1-a4e2b45f62d8 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:26:26.613636 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4be1f534-2d69-4248-baa1-a4e2b45f62d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:26:26.683379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1f534-2d69-4248-baa1-a4e2b45f62d8 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:26:26.688002 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d053a7d-a05d-4fe7-a86d-f751e9282941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:26:26.688878 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d053a7d-a05d-4fe7-a86d-f751e9282941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:26.689164 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d053a7d-a05d-4fe7-a86d-f751e9282941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:26.689440 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d053a7d-a05d-4fe7-a86d-f751e9282941 admin admin] PATH_INFO: `/v3/users/79357bc4bc0d4b8d9dd5b07a25510f3a` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:26.690629 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d053a7d-a05d-4fe7-a86d-f751e9282941 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79357bc4bc0d4b8d9dd5b07a25510f3a)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:26:26.692614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d053a7d-a05d-4fe7-a86d-f751e9282941 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:26:26.739844 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4d053a7d-a05d-4fe7-a86d-f751e9282941 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79357bc4bc0d4b8d9dd5b07a25510f3a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:26:26.752176 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4d053a7d-a05d-4fe7-a86d-f751e9282941 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79357bc4bc0d4b8d9dd5b07a25510f3a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:26:26.752886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4d053a7d-a05d-4fe7-a86d-f751e9282941 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79357bc4bc0d4b8d9dd5b07a25510f3a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:26:26.753500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4d053a7d-a05d-4fe7-a86d-f751e9282941 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79357bc4bc0d4b8d9dd5b07a25510f3a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:26:26.755525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:26:26.762615 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4d053a7d-a05d-4fe7-a86d-f751e9282941 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79357bc4bc0d4b8d9dd5b07a25510f3a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:26:26.775018 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1270/2532] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:26:26 2022] DELETE /identity/v3/users/79357bc4bc0d4b8d9dd5b07a25510f3a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:26:27.017721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba0c5d-00dd-427a-80f0-acda6772227c service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:26:27.030554 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba0c5d-00dd-427a-80f0-acda6772227c service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:26:27.099705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd87c2b2-02d1-400d-b338-1c224b79eec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:26:27.100568 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87c2b2-02d1-400d-b338-1c224b79eec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:27.100850 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87c2b2-02d1-400d-b338-1c224b79eec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:27.101121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87c2b2-02d1-400d-b338-1c224b79eec3 admin admin] PATH_INFO: `/v3/projects/59f9adafa99949b3b5f624c6ae7be57e` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:27.102305 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87c2b2-02d1-400d-b338-1c224b79eec3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59f9adafa99949b3b5f624c6ae7be57e)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:26:27.104264 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87c2b2-02d1-400d-b338-1c224b79eec3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:26:27.149054 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bd87c2b2-02d1-400d-b338-1c224b79eec3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59f9adafa99949b3b5f624c6ae7be57e', 'request_id': 'req-bd87c2b2-02d1-400d-b338-1c224b79eec3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:26:27.156745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bd87c2b2-02d1-400d-b338-1c224b79eec3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59f9adafa99949b3b5f624c6ae7be57e', 'request_id': 'req-bd87c2b2-02d1-400d-b338-1c224b79eec3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:26:27.162576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bd87c2b2-02d1-400d-b338-1c224b79eec3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59f9adafa99949b3b5f624c6ae7be57e', 'request_id': 'req-bd87c2b2-02d1-400d-b338-1c224b79eec3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:26:27.176303 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bd87c2b2-02d1-400d-b338-1c224b79eec3 admin admin] The token cache is being invalidate because project 59f9adafa99949b3b5f624c6ae7be57e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:26:27.176589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bd87c2b2-02d1-400d-b338-1c224b79eec3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:26:27.177352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bd87c2b2-02d1-400d-b338-1c224b79eec3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:26:27.178930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1263/2533] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:26:27 2022] DELETE /identity/v3/projects/59f9adafa99949b3b5f624c6ae7be57e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:26:27.184721 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f885b0-1ff6-4a4e-84a4-5d013ca86339 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:27.185008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f885b0-1ff6-4a4e-84a4-5d013ca86339 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:27.185284 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f885b0-1ff6-4a4e-84a4-5d013ca86339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:27.202909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-59f885b0-1ff6-4a4e-84a4-5d013ca86339 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:26:27.204165 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-59f885b0-1ff6-4a4e-84a4-5d013ca86339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:26:27.262477 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1271/2534] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:26:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:26:27.270034 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b866d3-70cb-41b8-a80d-af3bed546da4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:27.270320 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b866d3-70cb-41b8-a80d-af3bed546da4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:27.270596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b866d3-70cb-41b8-a80d-af3bed546da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:27.298217 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d3b866d3-70cb-41b8-a80d-af3bed546da4 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:26:27.299407 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d3b866d3-70cb-41b8-a80d-af3bed546da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:26:27.308269 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1264/2535] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:26:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:26:27.313300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f885b0-1ff6-4a4e-84a4-5d013ca86339 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:26:27.322893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f885b0-1ff6-4a4e-84a4-5d013ca86339 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:26:27.328337 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012ab35-68b5-421c-ac2b-514df242e973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:26:27.328989 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012ab35-68b5-421c-ac2b-514df242e973 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:27.329273 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012ab35-68b5-421c-ac2b-514df242e973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:27.329547 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012ab35-68b5-421c-ac2b-514df242e973 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:27.330271 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012ab35-68b5-421c-ac2b-514df242e973 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:26:27.330573 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012ab35-68b5-421c-ac2b-514df242e973 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:26:27.332190 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012ab35-68b5-421c-ac2b-514df242e973 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:26:27.343573 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1272/2536] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:26:27.348054 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b866d3-70cb-41b8-a80d-af3bed546da4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:26:27.357747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b866d3-70cb-41b8-a80d-af3bed546da4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:26:27.362382 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2094fa-6853-471e-9046-0ca45959b3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:26:27.363033 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2094fa-6853-471e-9046-0ca45959b3bf admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:27.363356 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2094fa-6853-471e-9046-0ca45959b3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:27.363636 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2094fa-6853-471e-9046-0ca45959b3bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:27.364318 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2094fa-6853-471e-9046-0ca45959b3bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:26:27.366209 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2094fa-6853-471e-9046-0ca45959b3bf admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:26:27.392706 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1265/2537] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:26:27 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:26:27.396637 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012ab35-68b5-421c-ac2b-514df242e973 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:26:27.406865 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012ab35-68b5-421c-ac2b-514df242e973 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:26:27.463620 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeeda7c-d13d-4da3-acdd-f01b16723e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:26:27.464291 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeeda7c-d13d-4da3-acdd-f01b16723e51 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:27.464575 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeeda7c-d13d-4da3-acdd-f01b16723e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:27.464853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeeda7c-d13d-4da3-acdd-f01b16723e51 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:27.465652 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeeda7c-d13d-4da3-acdd-f01b16723e51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:26:27.467629 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeeda7c-d13d-4da3-acdd-f01b16723e51 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:26:27.487634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1273/2538] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:26:27 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:26:27.491809 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2094fa-6853-471e-9046-0ca45959b3bf admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:26:27.502602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2094fa-6853-471e-9046-0ca45959b3bf admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:26:27.507141 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ad963-334b-4cf4-969d-5812f57ab4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:26:27.507817 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ad963-334b-4cf4-969d-5812f57ab4b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:27.508096 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ad963-334b-4cf4-969d-5812f57ab4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:27.508366 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ad963-334b-4cf4-969d-5812f57ab4b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:27.509112 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946ad963-334b-4cf4-969d-5812f57ab4b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:26:27.510681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946ad963-334b-4cf4-969d-5812f57ab4b2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:26:27.521619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1266/2539] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:26:27.525337 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeeda7c-d13d-4da3-acdd-f01b16723e51 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:26:27.535943 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeeda7c-d13d-4da3-acdd-f01b16723e51 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:26:27.540539 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f33912-31e0-4862-ab44-514baf588878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:26:27.541197 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f33912-31e0-4862-ab44-514baf588878 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:27.541480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f33912-31e0-4862-ab44-514baf588878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:27.541773 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f33912-31e0-4862-ab44-514baf588878 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:27.542480 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f33912-31e0-4862-ab44-514baf588878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:26:27.544919 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f33912-31e0-4862-ab44-514baf588878 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:26:27.554744 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1274/2540] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:26:27.558817 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ad963-334b-4cf4-969d-5812f57ab4b2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:26:27.568934 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ad963-334b-4cf4-969d-5812f57ab4b2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:26:27.573616 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5c021-64aa-44f1-890d-cc6930c219a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:26:27.574567 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5c021-64aa-44f1-890d-cc6930c219a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:27.575021 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5c021-64aa-44f1-890d-cc6930c219a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:27.575338 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5c021-64aa-44f1-890d-cc6930c219a5 admin admin] PATH_INFO: `/v3/projects/6ee52855d08e467f9d2f02685d7bf541/users/d0b828a3dca14f2283a16894364d6eef/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:27.589592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b5c021-64aa-44f1-890d-cc6930c219a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ee52855d08e467f9d2f02685d7bf541, user_id=d0b828a3dca14f2283a16894364d6eef, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:26:27.592957 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b5c021-64aa-44f1-890d-cc6930c219a5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:26:27.606733 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1267/2541] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:26:27 2022] PUT /identity/v3/projects/6ee52855d08e467f9d2f02685d7bf541/users/d0b828a3dca14f2283a16894364d6eef/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:26:27.612556 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f3e12-37b7-47d8-abe2-d00b5558c8a6 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:27.613035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f3e12-37b7-47d8-abe2-d00b5558c8a6 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:27.613167 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f3e12-37b7-47d8-abe2-d00b5558c8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:27.630870 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-7f4f3e12-37b7-47d8-abe2-d00b5558c8a6 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:26:27.632092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7f4f3e12-37b7-47d8-abe2-d00b5558c8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:26:27.676501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:26:27.691825 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1275/2542] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:26:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:26:27.701366 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5c021-64aa-44f1-890d-cc6930c219a5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:26:27.703072 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f4b5c021-64aa-44f1-890d-cc6930c219a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:26:27.717593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:26:27.763130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5c021-64aa-44f1-890d-cc6930c219a5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:26:27.766810 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e146c5-3ec4-4446-a2b9-a1ade7f54791 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:26:27.767441 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e146c5-3ec4-4446-a2b9-a1ade7f54791 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:27.767721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e146c5-3ec4-4446-a2b9-a1ade7f54791 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:27.767996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e146c5-3ec4-4446-a2b9-a1ade7f54791 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:27.776912 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e146c5-3ec4-4446-a2b9-a1ade7f54791 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:26:27.778877 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e146c5-3ec4-4446-a2b9-a1ade7f54791 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:26:27.788287 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1268/2543] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:26:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:26:29.440237 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f3e12-37b7-47d8-abe2-d00b5558c8a6 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:26:29.450848 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f3e12-37b7-47d8-abe2-d00b5558c8a6 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:26:29.454580 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230afc0-853f-4538-b6f9-28dd71598a65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:26:29.455355 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230afc0-853f-4538-b6f9-28dd71598a65 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:29.455729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230afc0-853f-4538-b6f9-28dd71598a65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:29.456013 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230afc0-853f-4538-b6f9-28dd71598a65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:29.457978 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9230afc0-853f-4538-b6f9-28dd71598a65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:26:29.521053 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9230afc0-853f-4538-b6f9-28dd71598a65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:26:29.522495 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9230afc0-853f-4538-b6f9-28dd71598a65 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:26:29.532740 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1276/2544] 10.222.0.81 () {60 vars in 1396 bytes} [Tue Jun 7 14:26:29 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3258 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:26:32.316766 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a61d04-b538-413d-83f6-c0e03415b673 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:32.317312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a61d04-b538-413d-83f6-c0e03415b673 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:32.318406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a61d04-b538-413d-83f6-c0e03415b673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:32.349007 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a4a61d04-b538-413d-83f6-c0e03415b673 None None] MFA Rules not processed for user `d0b828a3dca14f2283a16894364d6eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:26:32.351530 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a4a61d04-b538-413d-83f6-c0e03415b673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:26:32.419837 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1269/2545] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:26:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:26:32.424424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230afc0-853f-4538-b6f9-28dd71598a65 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:26:32.435585 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230afc0-853f-4538-b6f9-28dd71598a65 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:26:32.440189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b1ecd-e813-4096-b4c8-9ce3aa37665b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:26:32.440850 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b1ecd-e813-4096-b4c8-9ce3aa37665b admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:32.441133 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b1ecd-e813-4096-b4c8-9ce3aa37665b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:32.441406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b1ecd-e813-4096-b4c8-9ce3aa37665b admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:32.442103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b1ecd-e813-4096-b4c8-9ce3aa37665b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:26:32.444068 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b1ecd-e813-4096-b4c8-9ce3aa37665b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:26:32.474108 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1277/2546] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:26:32 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:26:32.478040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a61d04-b538-413d-83f6-c0e03415b673 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:26:32.491308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a61d04-b538-413d-83f6-c0e03415b673 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:26:32.558816 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f350781-bd8f-4564-a802-76a2cb0c19b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:26:32.559716 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f350781-bd8f-4564-a802-76a2cb0c19b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:32.559999 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f350781-bd8f-4564-a802-76a2cb0c19b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:32.560281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f350781-bd8f-4564-a802-76a2cb0c19b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:32.560970 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f350781-bd8f-4564-a802-76a2cb0c19b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:26:32.562892 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f350781-bd8f-4564-a802-76a2cb0c19b1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:26:32.584462 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1270/2547] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:26:32 2022] POST /identity/v3/users => generated 295 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:26:32.587726 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b1ecd-e813-4096-b4c8-9ce3aa37665b admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:26:32.598642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b1ecd-e813-4096-b4c8-9ce3aa37665b admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:26:32.603179 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f418bb4-8af9-41cc-be3e-baa22ab3387c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:26:32.604299 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f418bb4-8af9-41cc-be3e-baa22ab3387c admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:32.604299 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f418bb4-8af9-41cc-be3e-baa22ab3387c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:32.604509 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f418bb4-8af9-41cc-be3e-baa22ab3387c admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:32.605162 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f418bb4-8af9-41cc-be3e-baa22ab3387c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:26:32.607195 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f418bb4-8af9-41cc-be3e-baa22ab3387c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:26:32.619408 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1278/2548] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:26:32.623930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f350781-bd8f-4564-a802-76a2cb0c19b1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:26:32.634519 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f350781-bd8f-4564-a802-76a2cb0c19b1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:26:32.640013 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc39974-94ad-4a1f-803e-c07417bc1017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:26:32.640923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc39974-94ad-4a1f-803e-c07417bc1017 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:32.641202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc39974-94ad-4a1f-803e-c07417bc1017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:32.641483 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc39974-94ad-4a1f-803e-c07417bc1017 admin admin] PATH_INFO: `/v3/projects/25f284306301418aae8458f7fea5b9eb/users/fea498c8e7794e01a0bc5b4fb974229f/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:32.657746 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc39974-94ad-4a1f-803e-c07417bc1017 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25f284306301418aae8458f7fea5b9eb, user_id=fea498c8e7794e01a0bc5b4fb974229f, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:26:32.660605 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc39974-94ad-4a1f-803e-c07417bc1017 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:26:32.676226 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1271/2549] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:26:32 2022] PUT /identity/v3/projects/25f284306301418aae8458f7fea5b9eb/users/fea498c8e7794e01a0bc5b4fb974229f/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:26:32.680250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f418bb4-8af9-41cc-be3e-baa22ab3387c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:26:32.691731 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f418bb4-8af9-41cc-be3e-baa22ab3387c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:26:32.764910 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf1852-3689-4f1c-9d69-6b308d38442d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:26:32.765776 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf1852-3689-4f1c-9d69-6b308d38442d admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:32.766158 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf1852-3689-4f1c-9d69-6b308d38442d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:32.766572 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf1852-3689-4f1c-9d69-6b308d38442d admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:32.767405 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf1852-3689-4f1c-9d69-6b308d38442d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:26:32.769066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf1852-3689-4f1c-9d69-6b308d38442d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:26:32.782822 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1279/2550] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:26:32.784135 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc39974-94ad-4a1f-803e-c07417bc1017 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:26:32.795763 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc39974-94ad-4a1f-803e-c07417bc1017 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:26:32.800671 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88964d6d-df7a-43d9-ba9c-5352090d5d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:26:32.801573 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88964d6d-df7a-43d9-ba9c-5352090d5d0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:32.802138 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88964d6d-df7a-43d9-ba9c-5352090d5d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:32.802423 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88964d6d-df7a-43d9-ba9c-5352090d5d0b admin admin] PATH_INFO: `/v3/projects/25f284306301418aae8458f7fea5b9eb/users/fea498c8e7794e01a0bc5b4fb974229f/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:32.804487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88964d6d-df7a-43d9-ba9c-5352090d5d0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=25f284306301418aae8458f7fea5b9eb, user_id=fea498c8e7794e01a0bc5b4fb974229f, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:26:32.807400 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88964d6d-df7a-43d9-ba9c-5352090d5d0b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:26:32.813996 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:26:32.825074 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1272/2551] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:26:32 2022] PUT /identity/v3/projects/25f284306301418aae8458f7fea5b9eb/users/fea498c8e7794e01a0bc5b4fb974229f/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:26:34.850549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8c4c2-1b13-4dc3-b047-cea28b05d171 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:34.851094 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8c4c2-1b13-4dc3-b047-cea28b05d171 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:34.851210 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8c4c2-1b13-4dc3-b047-cea28b05d171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:34.876059 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-c1b8c4c2-1b13-4dc3-b047-cea28b05d171 None None] MFA Rules not processed for user `d20025ab12f4450da20c8e41b931b5fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:26:34.877296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c1b8c4c2-1b13-4dc3-b047-cea28b05d171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:26:34.941656 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:26:34.957574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1280/2552] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:26:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:26:37.826365 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2f133-9750-4b4d-849c-258c63a884ae None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:37.826921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2f133-9750-4b4d-849c-258c63a884ae None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:37.827138 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2f133-9750-4b4d-849c-258c63a884ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:37.864052 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-eec2f133-9750-4b4d-849c-258c63a884ae None None] MFA Rules not processed for user `fea498c8e7794e01a0bc5b4fb974229f`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:26:37.865394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-eec2f133-9750-4b4d-849c-258c63a884ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:26:37.944371 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1273/2553] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:26:37 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:26:37.959211 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8c4c2-1b13-4dc3-b047-cea28b05d171 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:26:37.960779 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c1b8c4c2-1b13-4dc3-b047-cea28b05d171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:26:38.039555 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8c4c2-1b13-4dc3-b047-cea28b05d171 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:26:38.045698 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12583cd5-e4ff-49b5-8c44-dc10b4409136 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:26:38.046362 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12583cd5-e4ff-49b5-8c44-dc10b4409136 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:38.046647 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12583cd5-e4ff-49b5-8c44-dc10b4409136 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:38.046932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12583cd5-e4ff-49b5-8c44-dc10b4409136 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:38.105256 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12583cd5-e4ff-49b5-8c44-dc10b4409136 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:26:38.106900 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12583cd5-e4ff-49b5-8c44-dc10b4409136 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:26:38.116009 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1281/2554] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:26:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:26:40.098901 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2f133-9750-4b4d-849c-258c63a884ae None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:26:40.100962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-eec2f133-9750-4b4d-849c-258c63a884ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:26:40.228275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2f133-9750-4b4d-849c-258c63a884ae None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:26:40.272675 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ffad9-87f3-4b2b-8d43-97e5b1de59e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:26:40.274151 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ffad9-87f3-4b2b-8d43-97e5b1de59e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:40.274442 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ffad9-87f3-4b2b-8d43-97e5b1de59e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:40.274720 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ffad9-87f3-4b2b-8d43-97e5b1de59e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:40.276785 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1e4ffad9-87f3-4b2b-8d43-97e5b1de59e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:26:40.287816 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ffad9-87f3-4b2b-8d43-97e5b1de59e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:26:40.289348 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ffad9-87f3-4b2b-8d43-97e5b1de59e5 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:26:40.300913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1274/2555] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:26:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:26:40.827393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12583cd5-e4ff-49b5-8c44-dc10b4409136 service nova] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:26:40.828987 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-12583cd5-e4ff-49b5-8c44-dc10b4409136 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:26:40.940659 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12583cd5-e4ff-49b5-8c44-dc10b4409136 service nova] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:26:40.950597 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1324863-c28c-4479-b1d8-6b2584a75cda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:26:40.952317 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1324863-c28c-4479-b1d8-6b2584a75cda service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:40.953086 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1324863-c28c-4479-b1d8-6b2584a75cda service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:40.953382 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1324863-c28c-4479-b1d8-6b2584a75cda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:40.955339 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e1324863-c28c-4479-b1d8-6b2584a75cda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:26:41.049529 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ffad9-87f3-4b2b-8d43-97e5b1de59e5 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:26:41.079808 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:26:41.082683 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ffad9-87f3-4b2b-8d43-97e5b1de59e5 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:26:41.106486 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1324863-c28c-4479-b1d8-6b2584a75cda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:26:41.108138 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1324863-c28c-4479-b1d8-6b2584a75cda service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:26:41.109336 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce2ded-bb5b-4bbb-86ba-b4d1654c0ec3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:26:41.112241 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce2ded-bb5b-4bbb-86ba-b4d1654c0ec3 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:41.113011 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce2ded-bb5b-4bbb-86ba-b4d1654c0ec3 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:41.113499 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce2ded-bb5b-4bbb-86ba-b4d1654c0ec3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:41.117037 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-01ce2ded-bb5b-4bbb-86ba-b4d1654c0ec3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:26:41.121616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1282/2556] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:26:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:26:41.139761 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce2ded-bb5b-4bbb-86ba-b4d1654c0ec3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:26:41.142739 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce2ded-bb5b-4bbb-86ba-b4d1654c0ec3 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:26:41.174892 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1275/2557] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:26:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:26:48.987457 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25718988-448e-4bab-8534-5ebb5c3d1808 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:48.988028 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25718988-448e-4bab-8534-5ebb5c3d1808 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:48.988028 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25718988-448e-4bab-8534-5ebb5c3d1808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:49.007620 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-25718988-448e-4bab-8534-5ebb5c3d1808 None None] MFA Rules not processed for user `6592aab6435242dca77b854a74ced4f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:26:49.008813 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-25718988-448e-4bab-8534-5ebb5c3d1808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:26:49.018730 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1283/2558] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:26:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:26:49.378123 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce2ded-bb5b-4bbb-86ba-b4d1654c0ec3 service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:26:49.392199 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce2ded-bb5b-4bbb-86ba-b4d1654c0ec3 service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:26:49.396379 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5fb301-8923-4d66-8ffc-80131a2db898 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:26:49.397088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fb301-8923-4d66-8ffc-80131a2db898 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:49.397389 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fb301-8923-4d66-8ffc-80131a2db898 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:49.397733 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fb301-8923-4d66-8ffc-80131a2db898 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:49.399692 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fb5fb301-8923-4d66-8ffc-80131a2db898 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:26:49.410340 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5fb301-8923-4d66-8ffc-80131a2db898 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:26:49.411914 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5fb301-8923-4d66-8ffc-80131a2db898 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:26:49.420100 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1276/2559] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:26:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:26:49.740133 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25718988-448e-4bab-8534-5ebb5c3d1808 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:26:49.741734 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-25718988-448e-4bab-8534-5ebb5c3d1808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:26:49.821858 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25718988-448e-4bab-8534-5ebb5c3d1808 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:26:49.827774 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff4656-1a55-4558-a642-80532f1cb658 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab73cf3febbf4c768cd0116306f16c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:26:49.828424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff4656-1a55-4558-a642-80532f1cb658 service placement] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:49.828701 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff4656-1a55-4558-a642-80532f1cb658 service placement] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:49.828975 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff4656-1a55-4558-a642-80532f1cb658 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:49.830780 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-76ff4656-1a55-4558-a642-80532f1cb658 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:26:49.842900 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ff4656-1a55-4558-a642-80532f1cb658 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:26:49.844776 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ff4656-1a55-4558-a642-80532f1cb658 service placement] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:26:49.854317 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1284/2560] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:26:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:26:57.871927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10d576-b366-46f2-9938-f6aa28c14696 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:57.872469 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10d576-b366-46f2-9938-f6aa28c14696 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:57.872469 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10d576-b366-46f2-9938-f6aa28c14696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:57.894777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-df10d576-b366-46f2-9938-f6aa28c14696 None None] MFA Rules not processed for user `d0b828a3dca14f2283a16894364d6eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:26:57.896983 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-df10d576-b366-46f2-9938-f6aa28c14696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:26:57.915317 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1277/2561] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:26:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:26:58.243797 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff4656-1a55-4558-a642-80532f1cb658 service placement] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:26:58.253838 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff4656-1a55-4558-a642-80532f1cb658 service placement] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:26:58.257433 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a489b8-fd66-411b-93fc-e2664b5b3f93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:26:58.258064 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a489b8-fd66-411b-93fc-e2664b5b3f93 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:26:58.258343 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a489b8-fd66-411b-93fc-e2664b5b3f93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:26:58.258614 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a489b8-fd66-411b-93fc-e2664b5b3f93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:26:58.260373 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-25a489b8-fd66-411b-93fc-e2664b5b3f93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:26:58.270358 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a489b8-fd66-411b-93fc-e2664b5b3f93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:26:58.271840 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a489b8-fd66-411b-93fc-e2664b5b3f93 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:26:58.280443 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1285/2562] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:26:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:27:01.875826 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10d576-b366-46f2-9938-f6aa28c14696 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:27:01.877291 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-df10d576-b366-46f2-9938-f6aa28c14696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:27:01.900709 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10d576-b366-46f2-9938-f6aa28c14696 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:27:01.909249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600003e-68ed-4719-9b46-59d25082115d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:27:01.910296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600003e-68ed-4719-9b46-59d25082115d service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:27:01.910793 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600003e-68ed-4719-9b46-59d25082115d service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:27:01.911808 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600003e-68ed-4719-9b46-59d25082115d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:27:01.914571 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1600003e-68ed-4719-9b46-59d25082115d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:27:01.929568 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600003e-68ed-4719-9b46-59d25082115d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:27:01.931332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600003e-68ed-4719-9b46-59d25082115d service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:27:01.942806 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1278/2563] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:27:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:27:08.276889 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=46 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:27:11.938523 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=82 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:27:11.941242 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a489b8-fd66-411b-93fc-e2664b5b3f93 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:27:11.956747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a489b8-fd66-411b-93fc-e2664b5b3f93 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:27:11.962239 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b37f10-d757-408c-b126-f42bb71ab0f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:27:11.962884 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b37f10-d757-408c-b126-f42bb71ab0f2 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:27:11.963739 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b37f10-d757-408c-b126-f42bb71ab0f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:27:11.964274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b37f10-d757-408c-b126-f42bb71ab0f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:27:11.966242 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-35b37f10-d757-408c-b126-f42bb71ab0f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:27:11.978945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b37f10-d757-408c-b126-f42bb71ab0f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:27:11.981381 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b37f10-d757-408c-b126-f42bb71ab0f2 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:27:11.993698 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1286/2564] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:27:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:27:14.925307 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600003e-68ed-4719-9b46-59d25082115d service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:27:14.935553 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600003e-68ed-4719-9b46-59d25082115d service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:27:14.939428 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87307293-9773-47a0-83ee-09ae810e3f7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:27:14.940065 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87307293-9773-47a0-83ee-09ae810e3f7d service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:27:14.940344 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87307293-9773-47a0-83ee-09ae810e3f7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:27:14.940621 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87307293-9773-47a0-83ee-09ae810e3f7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:27:14.950221 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87307293-9773-47a0-83ee-09ae810e3f7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:27:14.952016 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87307293-9773-47a0-83ee-09ae810e3f7d service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:27:14.961613 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1279/2565] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:27:14 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:27:21.988144 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=6 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:27:24.378371 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878ce53-7eb6-47dd-a397-f886df3f0108 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:27:24.379050 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878ce53-7eb6-47dd-a397-f886df3f0108 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:27:24.379050 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878ce53-7eb6-47dd-a397-f886df3f0108 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:27:24.380360 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1287/2566] 10.222.0.81 () {54 vars in 921 bytes} [Tue Jun 7 14:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:27:24.387735 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5f5c5-f327-436b-8146-12e69f1bcef5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:27:24.388240 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5f5c5-f327-436b-8146-12e69f1bcef5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:27:24.388359 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5f5c5-f327-436b-8146-12e69f1bcef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:27:24.408738 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5cb5f5c5-f327-436b-8146-12e69f1bcef5 None None] MFA Rules not processed for user `a6a08eaecbcd41228571739d8387882a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:27:24.409993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5cb5f5c5-f327-436b-8146-12e69f1bcef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:27:24.416590 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1280/2567] 10.222.0.81 () {58 vars in 1018 bytes} [Tue Jun 7 14:27:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:27:24.428797 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878ce53-7eb6-47dd-a397-f886df3f0108 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:27:24.440183 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878ce53-7eb6-47dd-a397-f886df3f0108 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:27:24.443973 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55951c07-5d0f-4e3e-a504-6a89b4ff1b4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:27:24.444688 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55951c07-5d0f-4e3e-a504-6a89b4ff1b4b service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:27:24.445042 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55951c07-5d0f-4e3e-a504-6a89b4ff1b4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:27:24.445425 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55951c07-5d0f-4e3e-a504-6a89b4ff1b4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:27:24.453638 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55951c07-5d0f-4e3e-a504-6a89b4ff1b4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:27:24.455065 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55951c07-5d0f-4e3e-a504-6a89b4ff1b4b service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:27:24.463034 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1288/2568] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:27:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:27:24.547141 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45639e-adfb-403f-b32e-cf0fda7df277 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:27:24.547459 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45639e-adfb-403f-b32e-cf0fda7df277 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:27:24.547738 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45639e-adfb-403f-b32e-cf0fda7df277 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:27:24.549126 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1281/2569] 10.222.0.81 () {54 vars in 921 bytes} [Tue Jun 7 14:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:27:24.553936 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b938148-4c53-49aa-bd04-adaa41863fdc None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:27:24.554224 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b938148-4c53-49aa-bd04-adaa41863fdc None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:27:24.554529 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b938148-4c53-49aa-bd04-adaa41863fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:27:24.590630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-1b938148-4c53-49aa-bd04-adaa41863fdc None None] MFA Rules not processed for user `2ae726fbc9cf40e68c7587552a118a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:27:24.591862 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1b938148-4c53-49aa-bd04-adaa41863fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:27:24.601098 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1289/2570] 10.222.0.81 () {58 vars in 1018 bytes} [Tue Jun 7 14:27:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:27:24.611418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa45639e-adfb-403f-b32e-cf0fda7df277 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:27:24.622735 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa45639e-adfb-403f-b32e-cf0fda7df277 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:27:24.626414 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7905f2e4-43c2-4b70-a050-989ef55d453c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:27:24.627041 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905f2e4-43c2-4b70-a050-989ef55d453c service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:27:24.627354 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905f2e4-43c2-4b70-a050-989ef55d453c service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:27:24.627627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905f2e4-43c2-4b70-a050-989ef55d453c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:27:24.638578 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7905f2e4-43c2-4b70-a050-989ef55d453c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:27:24.640095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7905f2e4-43c2-4b70-a050-989ef55d453c service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:27:24.651723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1282/2571] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:27:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:27:34.588139 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=15 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:27:34.647077 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=15 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:27:49.552617 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b938148-4c53-49aa-bd04-adaa41863fdc None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:27:49.564220 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b938148-4c53-49aa-bd04-adaa41863fdc None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:27:49.568367 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8ccc3-e61b-49ac-b7a2-f8c8eb5f3ef7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:27:49.569017 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8ccc3-e61b-49ac-b7a2-f8c8eb5f3ef7 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:27:49.569301 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8ccc3-e61b-49ac-b7a2-f8c8eb5f3ef7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:27:49.569577 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8ccc3-e61b-49ac-b7a2-f8c8eb5f3ef7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:27:49.571514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4fd8ccc3-e61b-49ac-b7a2-f8c8eb5f3ef7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:27:49.637623 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8ccc3-e61b-49ac-b7a2-f8c8eb5f3ef7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:27:49.639243 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8ccc3-e61b-49ac-b7a2-f8c8eb5f3ef7 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:27:49.649350 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1290/2572] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:27:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:27:54.681424 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7905f2e4-43c2-4b70-a050-989ef55d453c service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:27:54.682831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7905f2e4-43c2-4b70-a050-989ef55d453c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:27:54.698575 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7905f2e4-43c2-4b70-a050-989ef55d453c service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:27:54.703858 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfcbe6-23bd-402d-b935-55eb3cde5eed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab73cf3febbf4c768cd0116306f16c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:27:54.704502 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfcbe6-23bd-402d-b935-55eb3cde5eed service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:27:54.704780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfcbe6-23bd-402d-b935-55eb3cde5eed service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:27:54.705050 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfcbe6-23bd-402d-b935-55eb3cde5eed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:27:54.706792 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6ebfcbe6-23bd-402d-b935-55eb3cde5eed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:27:54.719351 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfcbe6-23bd-402d-b935-55eb3cde5eed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:27:54.721080 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfcbe6-23bd-402d-b935-55eb3cde5eed service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:27:54.733689 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1283/2573] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:27:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:27:54.744491 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8ccc3-e61b-49ac-b7a2-f8c8eb5f3ef7 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:27:54.747091 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4fd8ccc3-e61b-49ac-b7a2-f8c8eb5f3ef7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:27:54.768835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8ccc3-e61b-49ac-b7a2-f8c8eb5f3ef7 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:27:54.773661 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00135b12-a2de-47d2-976a-71920efc24ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:27:54.774549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00135b12-a2de-47d2-976a-71920efc24ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:27:54.774826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00135b12-a2de-47d2-976a-71920efc24ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:27:54.775301 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00135b12-a2de-47d2-976a-71920efc24ff admin admin] PATH_INFO: `/v3/users/d20025ab12f4450da20c8e41b931b5fc` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:27:54.776538 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00135b12-a2de-47d2-976a-71920efc24ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=d20025ab12f4450da20c8e41b931b5fc)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:27:54.779591 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00135b12-a2de-47d2-976a-71920efc24ff admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:27:54.832752 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-00135b12-a2de-47d2-976a-71920efc24ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd20025ab12f4450da20c8e41b931b5fc'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:27:54.845400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-00135b12-a2de-47d2-976a-71920efc24ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd20025ab12f4450da20c8e41b931b5fc'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:27:54.846063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-00135b12-a2de-47d2-976a-71920efc24ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd20025ab12f4450da20c8e41b931b5fc'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:27:54.846673 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-00135b12-a2de-47d2-976a-71920efc24ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd20025ab12f4450da20c8e41b931b5fc'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:27:54.853375 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-00135b12-a2de-47d2-976a-71920efc24ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd20025ab12f4450da20c8e41b931b5fc'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:27:54.867110 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1291/2574] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:27:54 2022] DELETE /identity/v3/users/d20025ab12f4450da20c8e41b931b5fc => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:27:55.042153 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfcbe6-23bd-402d-b935-55eb3cde5eed service placement] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:27:55.053682 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfcbe6-23bd-402d-b935-55eb3cde5eed service placement] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:27:55.116779 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0eccc-85ee-4683-a222-9282e1b7b884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:27:55.117641 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0eccc-85ee-4683-a222-9282e1b7b884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:27:55.117922 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0eccc-85ee-4683-a222-9282e1b7b884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:27:55.118193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0eccc-85ee-4683-a222-9282e1b7b884 admin admin] PATH_INFO: `/v3/projects/ec8e48464f6a4d1886aaf89cc8b28e5f` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:27:55.119403 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0eccc-85ee-4683-a222-9282e1b7b884 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec8e48464f6a4d1886aaf89cc8b28e5f)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:27:55.121343 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0eccc-85ee-4683-a222-9282e1b7b884 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:27:55.166283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ade0eccc-85ee-4683-a222-9282e1b7b884 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec8e48464f6a4d1886aaf89cc8b28e5f', 'request_id': 'req-ade0eccc-85ee-4683-a222-9282e1b7b884'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:27:55.173921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ade0eccc-85ee-4683-a222-9282e1b7b884 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec8e48464f6a4d1886aaf89cc8b28e5f', 'request_id': 'req-ade0eccc-85ee-4683-a222-9282e1b7b884'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:27:55.179402 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ade0eccc-85ee-4683-a222-9282e1b7b884 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec8e48464f6a4d1886aaf89cc8b28e5f', 'request_id': 'req-ade0eccc-85ee-4683-a222-9282e1b7b884'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:27:55.191087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ade0eccc-85ee-4683-a222-9282e1b7b884 admin admin] The token cache is being invalidate because project ec8e48464f6a4d1886aaf89cc8b28e5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:27:55.191338 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ade0eccc-85ee-4683-a222-9282e1b7b884 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:27:55.191959 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ade0eccc-85ee-4683-a222-9282e1b7b884 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:27:55.193476 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1284/2575] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:27:55 2022] DELETE /identity/v3/projects/ec8e48464f6a4d1886aaf89cc8b28e5f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:27:55.198321 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0004d-192e-4f0d-afa0-8f72cd6ec869 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:27:55.198605 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0004d-192e-4f0d-afa0-8f72cd6ec869 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:27:55.198877 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0004d-192e-4f0d-afa0-8f72cd6ec869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:27:55.218052 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-ece0004d-192e-4f0d-afa0-8f72cd6ec869 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:27:55.219264 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ece0004d-192e-4f0d-afa0-8f72cd6ec869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:27:55.277873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1292/2576] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:27:55.285832 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb75f6e-139a-4758-9bde-4f0649a817cc None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:27:55.286123 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb75f6e-139a-4758-9bde-4f0649a817cc None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:27:55.286398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb75f6e-139a-4758-9bde-4f0649a817cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:27:55.305332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6fb75f6e-139a-4758-9bde-4f0649a817cc None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:27:55.306651 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6fb75f6e-139a-4758-9bde-4f0649a817cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:27:55.315994 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1285/2577] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:27:55.320163 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0004d-192e-4f0d-afa0-8f72cd6ec869 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:27:55.330549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0004d-192e-4f0d-afa0-8f72cd6ec869 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:27:55.335186 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48c217-f1ff-45fb-8fc4-a73f801f582e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:27:55.335880 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48c217-f1ff-45fb-8fc4-a73f801f582e admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:27:55.336144 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48c217-f1ff-45fb-8fc4-a73f801f582e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:27:55.336414 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48c217-f1ff-45fb-8fc4-a73f801f582e admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:27:55.337136 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de48c217-f1ff-45fb-8fc4-a73f801f582e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:27:55.337438 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de48c217-f1ff-45fb-8fc4-a73f801f582e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:27:55.339005 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de48c217-f1ff-45fb-8fc4-a73f801f582e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:27:55.340825 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:27:55.353173 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1293/2578] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:27:55.357435 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb75f6e-139a-4758-9bde-4f0649a817cc None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:27:55.366750 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb75f6e-139a-4758-9bde-4f0649a817cc None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:27:55.372105 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac497b88-4961-4816-a0c7-5908fa79b707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:27:55.372967 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac497b88-4961-4816-a0c7-5908fa79b707 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:27:55.373244 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac497b88-4961-4816-a0c7-5908fa79b707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:27:55.373517 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac497b88-4961-4816-a0c7-5908fa79b707 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:27:55.374216 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac497b88-4961-4816-a0c7-5908fa79b707 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:27:55.376150 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac497b88-4961-4816-a0c7-5908fa79b707 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:27:55.411188 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1286/2579] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:27:55 2022] POST /identity/v3/projects => generated 389 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:27:55.415358 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48c217-f1ff-45fb-8fc4-a73f801f582e admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:27:55.425185 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48c217-f1ff-45fb-8fc4-a73f801f582e admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:27:55.482236 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ebc19f-8b2e-4e4c-8eea-3bb120f597a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:27:55.482929 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebc19f-8b2e-4e4c-8eea-3bb120f597a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:27:55.483209 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebc19f-8b2e-4e4c-8eea-3bb120f597a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:27:55.483511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebc19f-8b2e-4e4c-8eea-3bb120f597a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:27:55.484193 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ebc19f-8b2e-4e4c-8eea-3bb120f597a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:27:55.486133 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ebc19f-8b2e-4e4c-8eea-3bb120f597a1 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:27:55.507342 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1294/2580] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:27:55 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:27:55.512153 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac497b88-4961-4816-a0c7-5908fa79b707 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:27:55.522661 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac497b88-4961-4816-a0c7-5908fa79b707 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:27:55.527655 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df955a1-586d-4604-9999-2e6dbd244bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:27:55.528565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df955a1-586d-4604-9999-2e6dbd244bee admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:27:55.528913 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df955a1-586d-4604-9999-2e6dbd244bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:27:55.529097 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df955a1-586d-4604-9999-2e6dbd244bee admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:27:55.530026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df955a1-586d-4604-9999-2e6dbd244bee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:27:55.531638 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df955a1-586d-4604-9999-2e6dbd244bee admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:27:55.541382 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1287/2581] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:27:55.545962 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ebc19f-8b2e-4e4c-8eea-3bb120f597a1 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:27:55.555898 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ebc19f-8b2e-4e4c-8eea-3bb120f597a1 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:27:55.560526 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c30a1-cbcd-44cd-ad60-7883385872f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:27:55.561240 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c30a1-cbcd-44cd-ad60-7883385872f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:27:55.561527 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c30a1-cbcd-44cd-ad60-7883385872f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:27:55.561805 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c30a1-cbcd-44cd-ad60-7883385872f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:27:55.562526 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c30a1-cbcd-44cd-ad60-7883385872f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:27:55.564144 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c30a1-cbcd-44cd-ad60-7883385872f8 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:27:55.574416 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1295/2582] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:27:55.578305 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df955a1-586d-4604-9999-2e6dbd244bee admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:27:55.588370 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df955a1-586d-4604-9999-2e6dbd244bee admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:27:55.594231 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9b308-4659-447e-8462-88193dc7d8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:27:55.595106 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9b308-4659-447e-8462-88193dc7d8f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:27:55.595421 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9b308-4659-447e-8462-88193dc7d8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:27:55.595698 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9b308-4659-447e-8462-88193dc7d8f6 admin admin] PATH_INFO: `/v3/projects/4c6a6f6bdf7743cba1a36a3e8f67cf02/users/fefdd9dfa11641a598fcc29b2ad32afc/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:27:55.610179 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9b308-4659-447e-8462-88193dc7d8f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c6a6f6bdf7743cba1a36a3e8f67cf02, user_id=fefdd9dfa11641a598fcc29b2ad32afc, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:27:55.612905 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9b308-4659-447e-8462-88193dc7d8f6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:27:55.627121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1288/2583] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:27:55 2022] PUT /identity/v3/projects/4c6a6f6bdf7743cba1a36a3e8f67cf02/users/fefdd9dfa11641a598fcc29b2ad32afc/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:27:55.632261 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c3151-757e-4b64-8464-f113609c4944 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:27:55.632548 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c3151-757e-4b64-8464-f113609c4944 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:27:55.632824 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c3151-757e-4b64-8464-f113609c4944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:27:55.650882 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-786c3151-757e-4b64-8464-f113609c4944 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:27:55.652750 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-786c3151-757e-4b64-8464-f113609c4944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:27:55.709948 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1296/2584] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:27:55.720161 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9b308-4659-447e-8462-88193dc7d8f6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:27:55.721888 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-92f9b308-4659-447e-8462-88193dc7d8f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:27:55.728416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:27:55.781424 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9b308-4659-447e-8462-88193dc7d8f6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:27:55.785286 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8982b5b-4c51-4516-bc7a-e36ca86c634d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:27:55.785930 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8982b5b-4c51-4516-bc7a-e36ca86c634d service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:27:55.786210 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8982b5b-4c51-4516-bc7a-e36ca86c634d service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:27:55.786482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8982b5b-4c51-4516-bc7a-e36ca86c634d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:27:55.795627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8982b5b-4c51-4516-bc7a-e36ca86c634d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:27:55.797040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8982b5b-4c51-4516-bc7a-e36ca86c634d service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:27:55.806395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1289/2585] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:27:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:27:59.881739 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8435ca6-db82-4e24-a93c-41e72173c4b8 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:27:59.882615 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8435ca6-db82-4e24-a93c-41e72173c4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:27:59.883055 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8435ca6-db82-4e24-a93c-41e72173c4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:27:59.920940 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-b8435ca6-db82-4e24-a93c-41e72173c4b8 None None] MFA Rules not processed for user `fefdd9dfa11641a598fcc29b2ad32afc`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:27:59.922583 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b8435ca6-db82-4e24-a93c-41e72173c4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:27:59.997104 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1297/2586] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:27:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:28:00.014186 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8982b5b-4c51-4516-bc7a-e36ca86c634d service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:00.017095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d8982b5b-4c51-4516-bc7a-e36ca86c634d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:00.115812 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8982b5b-4c51-4516-bc7a-e36ca86c634d service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:00.123576 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6995d00-9662-43f3-b06f-df866dcebf18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:00.124243 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6995d00-9662-43f3-b06f-df866dcebf18 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:00.124524 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6995d00-9662-43f3-b06f-df866dcebf18 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:00.124801 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6995d00-9662-43f3-b06f-df866dcebf18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:00.135409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6995d00-9662-43f3-b06f-df866dcebf18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:00.138355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6995d00-9662-43f3-b06f-df866dcebf18 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:00.149865 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1290/2587] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:28:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:28:01.873623 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf721579-0277-448c-a8e2-5f6d0a8e0270 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:01.874131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf721579-0277-448c-a8e2-5f6d0a8e0270 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:01.874249 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf721579-0277-448c-a8e2-5f6d0a8e0270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:01.879946 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:28:01.893968 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-cf721579-0277-448c-a8e2-5f6d0a8e0270 None None] MFA Rules not processed for user `fefdd9dfa11641a598fcc29b2ad32afc`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:28:01.895174 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-cf721579-0277-448c-a8e2-5f6d0a8e0270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:01.903169 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1298/2588] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:28:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:28:05.359154 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b09f9-a168-4ffe-8921-d64107e7ae46 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:05.359689 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b09f9-a168-4ffe-8921-d64107e7ae46 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:05.359868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b09f9-a168-4ffe-8921-d64107e7ae46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:05.377064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-dd9b09f9-a168-4ffe-8921-d64107e7ae46 None None] MFA Rules not processed for user `d00b8bc311234e6da47b365ac8a7183a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:28:05.378360 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dd9b09f9-a168-4ffe-8921-d64107e7ae46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:05.433837 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1291/2589] 10.222.0.81 () {58 vars in 967 bytes} [Tue Jun 7 14:28:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:28:05.478057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf721579-0277-448c-a8e2-5f6d0a8e0270 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:05.480812 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-cf721579-0277-448c-a8e2-5f6d0a8e0270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:05.563855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf721579-0277-448c-a8e2-5f6d0a8e0270 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:05.569805 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e651c-1910-4dce-8343-9c4e08793a8e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da7c0f99060a44eaab743ba71f159e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:05.570698 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e651c-1910-4dce-8343-9c4e08793a8e service swift] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:05.571624 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e651c-1910-4dce-8343-9c4e08793a8e service swift] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:05.572072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e651c-1910-4dce-8343-9c4e08793a8e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:05.584504 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e651c-1910-4dce-8343-9c4e08793a8e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:05.586097 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e651c-1910-4dce-8343-9c4e08793a8e service swift] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:05.596269 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1299/2590] 10.222.0.81 () {60 vars in 1384 bytes} [Tue Jun 7 14:28:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:28:09.416495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b09f9-a168-4ffe-8921-d64107e7ae46 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:09.418089 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dd9b09f9-a168-4ffe-8921-d64107e7ae46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:09.478226 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:28:09.513418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b09f9-a168-4ffe-8921-d64107e7ae46 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:09.518554 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82de522-91b8-4a3d-9e75-d41d66ebe198 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab73cf3febbf4c768cd0116306f16c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:09.519251 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82de522-91b8-4a3d-9e75-d41d66ebe198 service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:09.519527 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82de522-91b8-4a3d-9e75-d41d66ebe198 service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:09.519804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82de522-91b8-4a3d-9e75-d41d66ebe198 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:09.521556 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f82de522-91b8-4a3d-9e75-d41d66ebe198 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:09.534955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82de522-91b8-4a3d-9e75-d41d66ebe198 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:09.536800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82de522-91b8-4a3d-9e75-d41d66ebe198 service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:09.549324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1292/2591] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:28:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:28:13.245311 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e651c-1910-4dce-8343-9c4e08793a8e service swift] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:13.246777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ca0e651c-1910-4dce-8343-9c4e08793a8e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:13.262222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e651c-1910-4dce-8343-9c4e08793a8e service swift] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:13.267194 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b43c65-575a-498c-a74f-666d34c290f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:13.268114 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b43c65-575a-498c-a74f-666d34c290f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:13.268397 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b43c65-575a-498c-a74f-666d34c290f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:13.268678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b43c65-575a-498c-a74f-666d34c290f8 admin admin] PATH_INFO: `/v3/users/9bb946aacaf145d5b718f82e28c999d9` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:13.269916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b43c65-575a-498c-a74f-666d34c290f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bb946aacaf145d5b718f82e28c999d9)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:13.271878 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b43c65-575a-498c-a74f-666d34c290f8 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:13.322508 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-61b43c65-575a-498c-a74f-666d34c290f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bb946aacaf145d5b718f82e28c999d9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:13.335459 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-61b43c65-575a-498c-a74f-666d34c290f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bb946aacaf145d5b718f82e28c999d9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:13.336223 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-61b43c65-575a-498c-a74f-666d34c290f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bb946aacaf145d5b718f82e28c999d9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:13.336963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-61b43c65-575a-498c-a74f-666d34c290f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bb946aacaf145d5b718f82e28c999d9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:13.344777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-61b43c65-575a-498c-a74f-666d34c290f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bb946aacaf145d5b718f82e28c999d9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:13.358806 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1300/2592] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:28:13 2022] DELETE /identity/v3/users/9bb946aacaf145d5b718f82e28c999d9 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:28:13.557965 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82de522-91b8-4a3d-9e75-d41d66ebe198 service placement] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:13.576059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82de522-91b8-4a3d-9e75-d41d66ebe198 service placement] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:13.638359 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92b652-8206-4837-a22b-77f644337be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:13.639239 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92b652-8206-4837-a22b-77f644337be6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:13.639531 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92b652-8206-4837-a22b-77f644337be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:13.639804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92b652-8206-4837-a22b-77f644337be6 admin admin] PATH_INFO: `/v3/projects/1c7b98807cae4a75968ed946de9c92bb` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:13.640992 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92b652-8206-4837-a22b-77f644337be6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c7b98807cae4a75968ed946de9c92bb)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:13.644010 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92b652-8206-4837-a22b-77f644337be6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:13.692896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ed92b652-8206-4837-a22b-77f644337be6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c7b98807cae4a75968ed946de9c92bb', 'request_id': 'req-ed92b652-8206-4837-a22b-77f644337be6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:13.707843 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ed92b652-8206-4837-a22b-77f644337be6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c7b98807cae4a75968ed946de9c92bb', 'request_id': 'req-ed92b652-8206-4837-a22b-77f644337be6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:13.715363 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ed92b652-8206-4837-a22b-77f644337be6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c7b98807cae4a75968ed946de9c92bb', 'request_id': 'req-ed92b652-8206-4837-a22b-77f644337be6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:13.727138 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ed92b652-8206-4837-a22b-77f644337be6 admin admin] The token cache is being invalidate because project 1c7b98807cae4a75968ed946de9c92bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:28:13.727472 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ed92b652-8206-4837-a22b-77f644337be6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:13.728124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ed92b652-8206-4837-a22b-77f644337be6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:13.729862 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1293/2593] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:28:13 2022] DELETE /identity/v3/projects/1c7b98807cae4a75968ed946de9c92bb => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:28:13.733800 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b43c65-575a-498c-a74f-666d34c290f8 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:13.735539 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-61b43c65-575a-498c-a74f-666d34c290f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:13.749368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b43c65-575a-498c-a74f-666d34c290f8 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:13.806674 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:28:13.813951 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa91a0-726c-4169-8755-3d0e1784faec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:13.814898 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa91a0-726c-4169-8755-3d0e1784faec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:13.815440 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa91a0-726c-4169-8755-3d0e1784faec admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:13.815911 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa91a0-726c-4169-8755-3d0e1784faec admin admin] PATH_INFO: `/v3/users/3b21bef5efa8422989c4426cc3594144` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:13.817248 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa91a0-726c-4169-8755-3d0e1784faec admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b21bef5efa8422989c4426cc3594144)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:13.819201 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa91a0-726c-4169-8755-3d0e1784faec admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:13.867721 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9aaa91a0-726c-4169-8755-3d0e1784faec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b21bef5efa8422989c4426cc3594144'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:13.879731 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9aaa91a0-726c-4169-8755-3d0e1784faec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b21bef5efa8422989c4426cc3594144'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:13.880406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9aaa91a0-726c-4169-8755-3d0e1784faec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b21bef5efa8422989c4426cc3594144'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:13.881007 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9aaa91a0-726c-4169-8755-3d0e1784faec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b21bef5efa8422989c4426cc3594144'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:13.888215 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9aaa91a0-726c-4169-8755-3d0e1784faec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b21bef5efa8422989c4426cc3594144'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:13.900399 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1301/2594] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:28:13 2022] DELETE /identity/v3/users/3b21bef5efa8422989c4426cc3594144 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:28:14.100276 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92b652-8206-4837-a22b-77f644337be6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:14.101736 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ed92b652-8206-4837-a22b-77f644337be6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:14.115962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92b652-8206-4837-a22b-77f644337be6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:14.179058 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:28:14.188750 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e8109-2d44-4681-81a2-99804d31cc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:14.190000 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e8109-2d44-4681-81a2-99804d31cc8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:14.190628 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e8109-2d44-4681-81a2-99804d31cc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:14.191178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e8109-2d44-4681-81a2-99804d31cc8c admin admin] PATH_INFO: `/v3/projects/2ce5417c58dd4bdd8598dd8dd2667a7d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:14.192298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e8109-2d44-4681-81a2-99804d31cc8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ce5417c58dd4bdd8598dd8dd2667a7d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:14.194241 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e8109-2d44-4681-81a2-99804d31cc8c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:14.240421 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-db1e8109-2d44-4681-81a2-99804d31cc8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ce5417c58dd4bdd8598dd8dd2667a7d', 'request_id': 'req-db1e8109-2d44-4681-81a2-99804d31cc8c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:14.247501 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-db1e8109-2d44-4681-81a2-99804d31cc8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ce5417c58dd4bdd8598dd8dd2667a7d', 'request_id': 'req-db1e8109-2d44-4681-81a2-99804d31cc8c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:14.253012 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-db1e8109-2d44-4681-81a2-99804d31cc8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ce5417c58dd4bdd8598dd8dd2667a7d', 'request_id': 'req-db1e8109-2d44-4681-81a2-99804d31cc8c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:14.265590 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-db1e8109-2d44-4681-81a2-99804d31cc8c admin admin] The token cache is being invalidate because project 2ce5417c58dd4bdd8598dd8dd2667a7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:28:14.265878 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-db1e8109-2d44-4681-81a2-99804d31cc8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:14.266733 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-db1e8109-2d44-4681-81a2-99804d31cc8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:14.268605 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1294/2595] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:28:14 2022] DELETE /identity/v3/projects/2ce5417c58dd4bdd8598dd8dd2667a7d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:28:22.674699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa91a0-726c-4169-8755-3d0e1784faec admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:22.676121 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9aaa91a0-726c-4169-8755-3d0e1784faec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:22.731906 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa91a0-726c-4169-8755-3d0e1784faec admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:22.780867 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec4c80-02c3-4e1e-9665-261592a08fe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:22.781440 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec4c80-02c3-4e1e-9665-261592a08fe1 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:22.781722 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec4c80-02c3-4e1e-9665-261592a08fe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:22.781994 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec4c80-02c3-4e1e-9665-261592a08fe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:22.783806 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a5ec4c80-02c3-4e1e-9665-261592a08fe1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:22.830442 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:28:22.855762 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec4c80-02c3-4e1e-9665-261592a08fe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:22.857233 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec4c80-02c3-4e1e-9665-261592a08fe1 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:22.866994 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1302/2596] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:28:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:28:24.261185 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=25,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:28:26.200331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e8109-2d44-4681-81a2-99804d31cc8c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:26.201796 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-db1e8109-2d44-4681-81a2-99804d31cc8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:26.214855 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e8109-2d44-4681-81a2-99804d31cc8c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:26.219990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55993a40-42f5-4c94-a5a4-5584b51fe06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:26.220867 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55993a40-42f5-4c94-a5a4-5584b51fe06c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:26.221150 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55993a40-42f5-4c94-a5a4-5584b51fe06c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:26.221426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55993a40-42f5-4c94-a5a4-5584b51fe06c admin admin] PATH_INFO: `/v3/users/6592aab6435242dca77b854a74ced4f2` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:26.222613 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55993a40-42f5-4c94-a5a4-5584b51fe06c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6592aab6435242dca77b854a74ced4f2)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:26.224680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55993a40-42f5-4c94-a5a4-5584b51fe06c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:26.270760 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-55993a40-42f5-4c94-a5a4-5584b51fe06c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6592aab6435242dca77b854a74ced4f2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:26.271432 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-55993a40-42f5-4c94-a5a4-5584b51fe06c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6592aab6435242dca77b854a74ced4f2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:26.272029 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-55993a40-42f5-4c94-a5a4-5584b51fe06c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6592aab6435242dca77b854a74ced4f2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:26.279180 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-55993a40-42f5-4c94-a5a4-5584b51fe06c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6592aab6435242dca77b854a74ced4f2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:26.290707 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-55993a40-42f5-4c94-a5a4-5584b51fe06c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6592aab6435242dca77b854a74ced4f2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:26.304991 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1295/2597] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:28:26 2022] DELETE /identity/v3/users/6592aab6435242dca77b854a74ced4f2 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:28:26.476207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec4c80-02c3-4e1e-9665-261592a08fe1 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:26.486927 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec4c80-02c3-4e1e-9665-261592a08fe1 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:26.553953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ada075-ec2a-4b39-a406-992eaee7ffaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:26.554829 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ada075-ec2a-4b39-a406-992eaee7ffaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:26.555104 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ada075-ec2a-4b39-a406-992eaee7ffaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:26.555404 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ada075-ec2a-4b39-a406-992eaee7ffaf admin admin] PATH_INFO: `/v3/projects/798b460fbb2e4be69df7f5f419844832` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:26.556594 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ada075-ec2a-4b39-a406-992eaee7ffaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=798b460fbb2e4be69df7f5f419844832)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:26.559629 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ada075-ec2a-4b39-a406-992eaee7ffaf admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:26.607139 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-25ada075-ec2a-4b39-a406-992eaee7ffaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '798b460fbb2e4be69df7f5f419844832', 'request_id': 'req-25ada075-ec2a-4b39-a406-992eaee7ffaf'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:26.615548 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-25ada075-ec2a-4b39-a406-992eaee7ffaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '798b460fbb2e4be69df7f5f419844832', 'request_id': 'req-25ada075-ec2a-4b39-a406-992eaee7ffaf'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:26.622783 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-25ada075-ec2a-4b39-a406-992eaee7ffaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '798b460fbb2e4be69df7f5f419844832', 'request_id': 'req-25ada075-ec2a-4b39-a406-992eaee7ffaf'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:26.633552 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-25ada075-ec2a-4b39-a406-992eaee7ffaf admin admin] The token cache is being invalidate because project 798b460fbb2e4be69df7f5f419844832 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:28:26.633838 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-25ada075-ec2a-4b39-a406-992eaee7ffaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:26.634479 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-25ada075-ec2a-4b39-a406-992eaee7ffaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:26.637015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1303/2598] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:28:26 2022] DELETE /identity/v3/projects/798b460fbb2e4be69df7f5f419844832 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:28:26.642052 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71a358-8cad-47cf-8e76-cb07bb18f301 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:26.642347 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71a358-8cad-47cf-8e76-cb07bb18f301 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:26.642645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71a358-8cad-47cf-8e76-cb07bb18f301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:26.661584 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7c71a358-8cad-47cf-8e76-cb07bb18f301 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:28:26.662796 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7c71a358-8cad-47cf-8e76-cb07bb18f301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:26.722674 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1296/2599] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:28:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:28:26.731398 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80878f-1b92-4bdb-ad6b-a640559e9157 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:26.731689 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80878f-1b92-4bdb-ad6b-a640559e9157 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:26.731965 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80878f-1b92-4bdb-ad6b-a640559e9157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:26.750652 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-de80878f-1b92-4bdb-ad6b-a640559e9157 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:28:26.751854 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-de80878f-1b92-4bdb-ad6b-a640559e9157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:26.759675 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1304/2600] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:28:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:28:26.764650 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71a358-8cad-47cf-8e76-cb07bb18f301 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:26.774330 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71a358-8cad-47cf-8e76-cb07bb18f301 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:26.778978 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b06d49-bb36-4f15-8e19-8a39f57367d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:26.779664 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b06d49-bb36-4f15-8e19-8a39f57367d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:26.779948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b06d49-bb36-4f15-8e19-8a39f57367d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:26.780223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b06d49-bb36-4f15-8e19-8a39f57367d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:26.780948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b06d49-bb36-4f15-8e19-8a39f57367d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:28:26.781258 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b06d49-bb36-4f15-8e19-8a39f57367d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:26.782831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b06d49-bb36-4f15-8e19-8a39f57367d1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:26.794797 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1297/2601] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:28:26.798837 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80878f-1b92-4bdb-ad6b-a640559e9157 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:26.808802 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80878f-1b92-4bdb-ad6b-a640559e9157 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:26.813455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a2076-90c1-44f2-b658-ba6d65898962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:26.814096 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a2076-90c1-44f2-b658-ba6d65898962 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:26.814373 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a2076-90c1-44f2-b658-ba6d65898962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:26.814675 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a2076-90c1-44f2-b658-ba6d65898962 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:26.815393 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a2076-90c1-44f2-b658-ba6d65898962 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:26.817270 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a2076-90c1-44f2-b658-ba6d65898962 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:26.843483 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1305/2602] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:28:26 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:28:26.847511 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b06d49-bb36-4f15-8e19-8a39f57367d1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:26.857420 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b06d49-bb36-4f15-8e19-8a39f57367d1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:26.912425 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b4181-2c10-4231-9747-32f9ca305ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:26.913084 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b4181-2c10-4231-9747-32f9ca305ac4 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:26.913363 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b4181-2c10-4231-9747-32f9ca305ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:26.913634 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b4181-2c10-4231-9747-32f9ca305ac4 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:26.914307 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b4181-2c10-4231-9747-32f9ca305ac4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:26.916308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b4181-2c10-4231-9747-32f9ca305ac4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:26.923040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:28:26.938223 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1298/2603] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:28:26 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:28:26.942301 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a2076-90c1-44f2-b658-ba6d65898962 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:26.953651 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a2076-90c1-44f2-b658-ba6d65898962 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:26.958215 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb38705-cb20-4623-a73d-b3d008830603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:26.958866 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb38705-cb20-4623-a73d-b3d008830603 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:26.959145 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb38705-cb20-4623-a73d-b3d008830603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:26.959451 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb38705-cb20-4623-a73d-b3d008830603 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:26.960170 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb38705-cb20-4623-a73d-b3d008830603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:26.961733 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb38705-cb20-4623-a73d-b3d008830603 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:26.972958 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1306/2604] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:28:26.976839 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b4181-2c10-4231-9747-32f9ca305ac4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:26.987285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b4181-2c10-4231-9747-32f9ca305ac4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:26.991840 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdca88d-e2e1-4ebe-8049-88f63c5caeb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:26.992495 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdca88d-e2e1-4ebe-8049-88f63c5caeb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:26.992779 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdca88d-e2e1-4ebe-8049-88f63c5caeb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:26.993056 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdca88d-e2e1-4ebe-8049-88f63c5caeb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:26.993780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdca88d-e2e1-4ebe-8049-88f63c5caeb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:26.996161 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdca88d-e2e1-4ebe-8049-88f63c5caeb5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:27.006011 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1299/2605] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:28:27.010077 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb38705-cb20-4623-a73d-b3d008830603 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:27.020972 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb38705-cb20-4623-a73d-b3d008830603 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:27.025949 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfee16b-2f3e-4e7f-92c5-a7becca54c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:27.026873 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfee16b-2f3e-4e7f-92c5-a7becca54c9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:27.027252 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfee16b-2f3e-4e7f-92c5-a7becca54c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:27.027525 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfee16b-2f3e-4e7f-92c5-a7becca54c9f admin admin] PATH_INFO: `/v3/projects/1d48c869d2264d238dc3aa322386a4ee/users/58effe9c898b4f6da76915b2c3ca8b70/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:27.043407 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfee16b-2f3e-4e7f-92c5-a7becca54c9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d48c869d2264d238dc3aa322386a4ee, user_id=58effe9c898b4f6da76915b2c3ca8b70, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:27.046095 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfee16b-2f3e-4e7f-92c5-a7becca54c9f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:27.059783 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1307/2606] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:28:27 2022] PUT /identity/v3/projects/1d48c869d2264d238dc3aa322386a4ee/users/58effe9c898b4f6da76915b2c3ca8b70/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:28:27.070016 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff503df3-c820-4a73-a928-4e3712ba866b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:27.070310 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff503df3-c820-4a73-a928-4e3712ba866b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:27.070590 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff503df3-c820-4a73-a928-4e3712ba866b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:27.097365 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ff503df3-c820-4a73-a928-4e3712ba866b None None] MFA Rules not processed for user `58effe9c898b4f6da76915b2c3ca8b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:28:27.098577 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ff503df3-c820-4a73-a928-4e3712ba866b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:27.165141 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1300/2607] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:28:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:28:27.169856 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfee16b-2f3e-4e7f-92c5-a7becca54c9f admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:27.179854 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:28:27.233051 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfee16b-2f3e-4e7f-92c5-a7becca54c9f admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:27.237647 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76630e01-b0e3-4048-afad-ffd0e3019007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:27.238325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76630e01-b0e3-4048-afad-ffd0e3019007 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:27.238597 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76630e01-b0e3-4048-afad-ffd0e3019007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:27.238874 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76630e01-b0e3-4048-afad-ffd0e3019007 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:27.239598 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76630e01-b0e3-4048-afad-ffd0e3019007 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:27.241534 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76630e01-b0e3-4048-afad-ffd0e3019007 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:27.267236 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1308/2608] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:28:27 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:28:27.271358 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff503df3-c820-4a73-a928-4e3712ba866b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:27.282283 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff503df3-c820-4a73-a928-4e3712ba866b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:27.337463 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23119ca7-ca12-43d2-8d72-3a76d6e28d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:27.338149 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23119ca7-ca12-43d2-8d72-3a76d6e28d2c admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:27.338435 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23119ca7-ca12-43d2-8d72-3a76d6e28d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:27.338731 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23119ca7-ca12-43d2-8d72-3a76d6e28d2c admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:27.339466 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23119ca7-ca12-43d2-8d72-3a76d6e28d2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:27.341654 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23119ca7-ca12-43d2-8d72-3a76d6e28d2c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:27.361626 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1301/2609] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:28:27 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:28:27.365712 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76630e01-b0e3-4048-afad-ffd0e3019007 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:27.376453 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76630e01-b0e3-4048-afad-ffd0e3019007 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:27.381641 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa775ed4-98a9-40b9-aa02-2a7e98e94a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:27.382304 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa775ed4-98a9-40b9-aa02-2a7e98e94a09 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:27.382588 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa775ed4-98a9-40b9-aa02-2a7e98e94a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:27.382865 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa775ed4-98a9-40b9-aa02-2a7e98e94a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:27.383632 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa775ed4-98a9-40b9-aa02-2a7e98e94a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:27.385214 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa775ed4-98a9-40b9-aa02-2a7e98e94a09 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:27.396136 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1309/2610] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:28:27.400014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23119ca7-ca12-43d2-8d72-3a76d6e28d2c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:27.410449 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23119ca7-ca12-43d2-8d72-3a76d6e28d2c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:27.415024 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28dba2-c9b3-47a0-881c-96129e5c2265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:27.415718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28dba2-c9b3-47a0-881c-96129e5c2265 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:27.416003 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28dba2-c9b3-47a0-881c-96129e5c2265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:27.416284 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28dba2-c9b3-47a0-881c-96129e5c2265 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:27.417747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d28dba2-c9b3-47a0-881c-96129e5c2265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:27.419354 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d28dba2-c9b3-47a0-881c-96129e5c2265 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:27.429091 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1302/2611] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:28:27.433106 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa775ed4-98a9-40b9-aa02-2a7e98e94a09 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:27.442840 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa775ed4-98a9-40b9-aa02-2a7e98e94a09 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:27.447470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d0f57-79eb-468b-ae62-a5937d065de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:27.448338 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d0f57-79eb-468b-ae62-a5937d065de4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:27.448615 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d0f57-79eb-468b-ae62-a5937d065de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:27.448887 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d0f57-79eb-468b-ae62-a5937d065de4 admin admin] PATH_INFO: `/v3/projects/f323c9706dbf4d7aa5f5e8acaff74dfa/users/debed890e7c54ce187076691cf61e525/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:27.463013 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d0f57-79eb-468b-ae62-a5937d065de4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f323c9706dbf4d7aa5f5e8acaff74dfa, user_id=debed890e7c54ce187076691cf61e525, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:27.466526 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d0f57-79eb-468b-ae62-a5937d065de4 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:27.480796 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1310/2612] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:28:27 2022] PUT /identity/v3/projects/f323c9706dbf4d7aa5f5e8acaff74dfa/users/debed890e7c54ce187076691cf61e525/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:28:27.490920 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d346a-b337-4532-a795-cd0ce8a32573 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:27.491153 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d346a-b337-4532-a795-cd0ce8a32573 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:27.491462 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d346a-b337-4532-a795-cd0ce8a32573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:27.517138 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-da5d346a-b337-4532-a795-cd0ce8a32573 None None] MFA Rules not processed for user `debed890e7c54ce187076691cf61e525`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:28:27.518349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-da5d346a-b337-4532-a795-cd0ce8a32573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:27.552315 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:28:27.584107 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1303/2613] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:28:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:28:27.589128 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d0f57-79eb-468b-ae62-a5937d065de4 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:27.650660 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d0f57-79eb-468b-ae62-a5937d065de4 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:27.656098 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0fe56-abe2-4236-b187-ea0e8428a4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:27.656755 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0fe56-abe2-4236-b187-ea0e8428a4c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:27.657038 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0fe56-abe2-4236-b187-ea0e8428a4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:27.657314 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0fe56-abe2-4236-b187-ea0e8428a4c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:27.658008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a0fe56-abe2-4236-b187-ea0e8428a4c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:27.659944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a0fe56-abe2-4236-b187-ea0e8428a4c5 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:27.687017 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1311/2614] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:28:27 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:28:27.690112 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d346a-b337-4532-a795-cd0ce8a32573 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:27.700764 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d346a-b337-4532-a795-cd0ce8a32573 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:27.755699 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcbd05-e5b5-4d90-a168-0434a23e8c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:27.756288 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcbd05-e5b5-4d90-a168-0434a23e8c30 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:27.756568 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcbd05-e5b5-4d90-a168-0434a23e8c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:27.756838 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcbd05-e5b5-4d90-a168-0434a23e8c30 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:27.757509 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bcbd05-e5b5-4d90-a168-0434a23e8c30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:27.759417 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bcbd05-e5b5-4d90-a168-0434a23e8c30 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:27.780175 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1304/2615] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:28:27 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:28:27.784219 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0fe56-abe2-4236-b187-ea0e8428a4c5 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:27.795293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0fe56-abe2-4236-b187-ea0e8428a4c5 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:27.799897 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48f830-ce15-4d40-be8a-cc3a858e30a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:27.800554 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48f830-ce15-4d40-be8a-cc3a858e30a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:27.800835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48f830-ce15-4d40-be8a-cc3a858e30a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:27.801109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48f830-ce15-4d40-be8a-cc3a858e30a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:27.801853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d48f830-ce15-4d40-be8a-cc3a858e30a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:27.803455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d48f830-ce15-4d40-be8a-cc3a858e30a0 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:27.813831 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1312/2616] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:28:27.817865 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcbd05-e5b5-4d90-a168-0434a23e8c30 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:27.828386 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcbd05-e5b5-4d90-a168-0434a23e8c30 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:27.832974 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c9f45-dce0-4cf8-ba6a-db7e657cff43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:27.833851 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c9f45-dce0-4cf8-ba6a-db7e657cff43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:27.834131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c9f45-dce0-4cf8-ba6a-db7e657cff43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:27.834405 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c9f45-dce0-4cf8-ba6a-db7e657cff43 admin admin] PATH_INFO: `/v3/projects/1459c1065a0c452b8ee269410c89104b/users/1ed7d867aed34eeda1dd1737269c329d/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:27.850492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c9f45-dce0-4cf8-ba6a-db7e657cff43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1459c1065a0c452b8ee269410c89104b, user_id=1ed7d867aed34eeda1dd1737269c329d, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:27.853182 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c9f45-dce0-4cf8-ba6a-db7e657cff43 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:27.867154 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1305/2617] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:28:27 2022] PUT /identity/v3/projects/1459c1065a0c452b8ee269410c89104b/users/1ed7d867aed34eeda1dd1737269c329d/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:28:27.871917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48f830-ce15-4d40-be8a-cc3a858e30a0 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:27.881794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48f830-ce15-4d40-be8a-cc3a858e30a0 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:27.911865 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:28:27.938397 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a8bad-b3e9-4819-9b62-a10fd266b189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:27.939059 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a8bad-b3e9-4819-9b62-a10fd266b189 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:27.939376 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a8bad-b3e9-4819-9b62-a10fd266b189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:27.939653 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a8bad-b3e9-4819-9b62-a10fd266b189 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:27.940369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a8bad-b3e9-4819-9b62-a10fd266b189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:27.941942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a8bad-b3e9-4819-9b62-a10fd266b189 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:27.952148 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1313/2618] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:28:27.956505 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c9f45-dce0-4cf8-ba6a-db7e657cff43 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:27.970871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c9f45-dce0-4cf8-ba6a-db7e657cff43 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:27.975686 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4253b-9e18-4ad7-997d-016e49158dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:27.976569 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4253b-9e18-4ad7-997d-016e49158dfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:27.976853 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4253b-9e18-4ad7-997d-016e49158dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:27.977128 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4253b-9e18-4ad7-997d-016e49158dfe admin admin] PATH_INFO: `/v3/projects/1459c1065a0c452b8ee269410c89104b/users/1ed7d867aed34eeda1dd1737269c329d/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:27.979439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4253b-9e18-4ad7-997d-016e49158dfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=1459c1065a0c452b8ee269410c89104b, user_id=1ed7d867aed34eeda1dd1737269c329d, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:27.982147 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4253b-9e18-4ad7-997d-016e49158dfe admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:27.992695 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1306/2619] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:28:27 2022] PUT /identity/v3/projects/1459c1065a0c452b8ee269410c89104b/users/1ed7d867aed34eeda1dd1737269c329d/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:28:28.004703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d79b07-454e-4768-b63a-90919732d7c9 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:28.004991 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d79b07-454e-4768-b63a-90919732d7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:28.005267 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d79b07-454e-4768-b63a-90919732d7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:28.032962 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-92d79b07-454e-4768-b63a-90919732d7c9 None None] MFA Rules not processed for user `1ed7d867aed34eeda1dd1737269c329d`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:28:28.034097 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-92d79b07-454e-4768-b63a-90919732d7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:28.102353 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1314/2620] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:28:28 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:28:28.113654 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4253b-9e18-4ad7-997d-016e49158dfe admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:28.115433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7ab4253b-9e18-4ad7-997d-016e49158dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:28.175499 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4253b-9e18-4ad7-997d-016e49158dfe admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:28.179030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82cd31-a112-4c52-954f-be351e594df2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:28.179708 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82cd31-a112-4c52-954f-be351e594df2 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:28.179989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82cd31-a112-4c52-954f-be351e594df2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:28.180259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82cd31-a112-4c52-954f-be351e594df2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:28.189859 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed82cd31-a112-4c52-954f-be351e594df2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:28.191297 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed82cd31-a112-4c52-954f-be351e594df2 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:28.201344 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1307/2621] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:28:28 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:28:28.240158 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d79b07-454e-4768-b63a-90919732d7c9 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:28.241560 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-92d79b07-454e-4768-b63a-90919732d7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:28.254299 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d79b07-454e-4768-b63a-90919732d7c9 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:28.257939 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12113cf-3e88-4171-b58b-d20338d4c84e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:28.258590 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12113cf-3e88-4171-b58b-d20338d4c84e service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:28.258882 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12113cf-3e88-4171-b58b-d20338d4c84e service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:28.259156 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12113cf-3e88-4171-b58b-d20338d4c84e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:28.317107 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12113cf-3e88-4171-b58b-d20338d4c84e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:28.318460 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12113cf-3e88-4171-b58b-d20338d4c84e service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:28.328095 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1315/2622] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:28:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:28:31.199863 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82cd31-a112-4c52-954f-be351e594df2 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:31.233775 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:28:31.262918 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82cd31-a112-4c52-954f-be351e594df2 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:31.267515 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7547026-9152-4df8-a31e-6a71e1b73306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:31.268369 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7547026-9152-4df8-a31e-6a71e1b73306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:31.268652 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7547026-9152-4df8-a31e-6a71e1b73306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:31.268928 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7547026-9152-4df8-a31e-6a71e1b73306 admin admin] PATH_INFO: `/v3/users/58effe9c898b4f6da76915b2c3ca8b70` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:31.270115 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7547026-9152-4df8-a31e-6a71e1b73306 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58effe9c898b4f6da76915b2c3ca8b70)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:31.272140 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7547026-9152-4df8-a31e-6a71e1b73306 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:31.317401 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b7547026-9152-4df8-a31e-6a71e1b73306 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58effe9c898b4f6da76915b2c3ca8b70'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:31.318051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b7547026-9152-4df8-a31e-6a71e1b73306 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58effe9c898b4f6da76915b2c3ca8b70'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:31.318655 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b7547026-9152-4df8-a31e-6a71e1b73306 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58effe9c898b4f6da76915b2c3ca8b70'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:31.326034 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b7547026-9152-4df8-a31e-6a71e1b73306 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58effe9c898b4f6da76915b2c3ca8b70'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:31.337469 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b7547026-9152-4df8-a31e-6a71e1b73306 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58effe9c898b4f6da76915b2c3ca8b70'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:31.350554 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1308/2623] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:28:31 2022] DELETE /identity/v3/users/58effe9c898b4f6da76915b2c3ca8b70 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:28:31.355098 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e62fbb-10d5-4f53-9508-9cf39d154c05 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:31.355627 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e62fbb-10d5-4f53-9508-9cf39d154c05 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:31.355739 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e62fbb-10d5-4f53-9508-9cf39d154c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:31.373657 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-74e62fbb-10d5-4f53-9508-9cf39d154c05 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:28:31.374796 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-74e62fbb-10d5-4f53-9508-9cf39d154c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:31.420645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:28:31.433372 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1316/2624] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:28:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:28:31.443140 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7547026-9152-4df8-a31e-6a71e1b73306 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:31.444565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b7547026-9152-4df8-a31e-6a71e1b73306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:31.501169 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7547026-9152-4df8-a31e-6a71e1b73306 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:31.504817 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b2cf4-33a5-4b09-8613-04b8d2223448 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:31.505462 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b2cf4-33a5-4b09-8613-04b8d2223448 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:31.505745 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b2cf4-33a5-4b09-8613-04b8d2223448 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:31.506022 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b2cf4-33a5-4b09-8613-04b8d2223448 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:31.516407 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b2cf4-33a5-4b09-8613-04b8d2223448 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:31.517868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b2cf4-33a5-4b09-8613-04b8d2223448 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:31.528052 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1309/2625] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:28:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:28:31.798616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e62fbb-10d5-4f53-9508-9cf39d154c05 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:31.800061 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-74e62fbb-10d5-4f53-9508-9cf39d154c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:31.812237 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e62fbb-10d5-4f53-9508-9cf39d154c05 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:31.816741 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca93621-b7e9-4774-999c-411bd4d831fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:31.817605 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca93621-b7e9-4774-999c-411bd4d831fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:31.817888 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca93621-b7e9-4774-999c-411bd4d831fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:31.818161 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca93621-b7e9-4774-999c-411bd4d831fe admin admin] PATH_INFO: `/v3/projects/1d48c869d2264d238dc3aa322386a4ee` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:31.819448 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca93621-b7e9-4774-999c-411bd4d831fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d48c869d2264d238dc3aa322386a4ee)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:31.821346 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca93621-b7e9-4774-999c-411bd4d831fe admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:31.868725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6ca93621-b7e9-4774-999c-411bd4d831fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d48c869d2264d238dc3aa322386a4ee', 'request_id': 'req-6ca93621-b7e9-4774-999c-411bd4d831fe'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:31.875919 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6ca93621-b7e9-4774-999c-411bd4d831fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d48c869d2264d238dc3aa322386a4ee', 'request_id': 'req-6ca93621-b7e9-4774-999c-411bd4d831fe'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:31.881970 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6ca93621-b7e9-4774-999c-411bd4d831fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d48c869d2264d238dc3aa322386a4ee', 'request_id': 'req-6ca93621-b7e9-4774-999c-411bd4d831fe'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:31.893028 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6ca93621-b7e9-4774-999c-411bd4d831fe admin admin] The token cache is being invalidate because project 1d48c869d2264d238dc3aa322386a4ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:28:31.893315 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6ca93621-b7e9-4774-999c-411bd4d831fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:31.893944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6ca93621-b7e9-4774-999c-411bd4d831fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:31.895515 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1317/2626] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:28:31 2022] DELETE /identity/v3/projects/1d48c869d2264d238dc3aa322386a4ee => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:28:31.899698 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b2cf4-33a5-4b09-8613-04b8d2223448 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:31.909942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b2cf4-33a5-4b09-8613-04b8d2223448 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:31.956876 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:28:31.967787 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4ad1c-2525-4bf3-a4d3-2b5a321dc0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:31.968685 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4ad1c-2525-4bf3-a4d3-2b5a321dc0e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:31.968962 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4ad1c-2525-4bf3-a4d3-2b5a321dc0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:31.969231 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4ad1c-2525-4bf3-a4d3-2b5a321dc0e5 admin admin] PATH_INFO: `/v3/users/debed890e7c54ce187076691cf61e525` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:31.970399 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4ad1c-2525-4bf3-a4d3-2b5a321dc0e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=debed890e7c54ce187076691cf61e525)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:31.972357 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4ad1c-2525-4bf3-a4d3-2b5a321dc0e5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:32.017597 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-abb4ad1c-2525-4bf3-a4d3-2b5a321dc0e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'debed890e7c54ce187076691cf61e525'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:32.018238 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-abb4ad1c-2525-4bf3-a4d3-2b5a321dc0e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'debed890e7c54ce187076691cf61e525'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:32.018838 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-abb4ad1c-2525-4bf3-a4d3-2b5a321dc0e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'debed890e7c54ce187076691cf61e525'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:32.029118 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-abb4ad1c-2525-4bf3-a4d3-2b5a321dc0e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'debed890e7c54ce187076691cf61e525'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:32.040964 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-abb4ad1c-2525-4bf3-a4d3-2b5a321dc0e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'debed890e7c54ce187076691cf61e525'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:32.054215 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1310/2627] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:28:31 2022] DELETE /identity/v3/users/debed890e7c54ce187076691cf61e525 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:28:32.327029 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca93621-b7e9-4774-999c-411bd4d831fe admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:32.328586 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6ca93621-b7e9-4774-999c-411bd4d831fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:32.342676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca93621-b7e9-4774-999c-411bd4d831fe admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:32.399732 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152c769-e846-457a-ab08-c0029a41c954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:32.400635 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152c769-e846-457a-ab08-c0029a41c954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:32.401062 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152c769-e846-457a-ab08-c0029a41c954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:32.401816 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152c769-e846-457a-ab08-c0029a41c954 admin admin] PATH_INFO: `/v3/projects/f323c9706dbf4d7aa5f5e8acaff74dfa` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:32.403074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7152c769-e846-457a-ab08-c0029a41c954 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f323c9706dbf4d7aa5f5e8acaff74dfa)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:32.405032 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7152c769-e846-457a-ab08-c0029a41c954 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:32.449993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7152c769-e846-457a-ab08-c0029a41c954 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f323c9706dbf4d7aa5f5e8acaff74dfa', 'request_id': 'req-7152c769-e846-457a-ab08-c0029a41c954'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:32.457350 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7152c769-e846-457a-ab08-c0029a41c954 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f323c9706dbf4d7aa5f5e8acaff74dfa', 'request_id': 'req-7152c769-e846-457a-ab08-c0029a41c954'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:32.464268 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7152c769-e846-457a-ab08-c0029a41c954 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f323c9706dbf4d7aa5f5e8acaff74dfa', 'request_id': 'req-7152c769-e846-457a-ab08-c0029a41c954'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:32.474835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7152c769-e846-457a-ab08-c0029a41c954 admin admin] The token cache is being invalidate because project f323c9706dbf4d7aa5f5e8acaff74dfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:28:32.475127 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7152c769-e846-457a-ab08-c0029a41c954 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:32.475797 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7152c769-e846-457a-ab08-c0029a41c954 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:32.477325 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1318/2628] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:28:32 2022] DELETE /identity/v3/projects/f323c9706dbf4d7aa5f5e8acaff74dfa => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:28:32.481318 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4ad1c-2525-4bf3-a4d3-2b5a321dc0e5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:32.482719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-abb4ad1c-2525-4bf3-a4d3-2b5a321dc0e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:32.495617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4ad1c-2525-4bf3-a4d3-2b5a321dc0e5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:32.552289 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e57673-dd5e-40f8-a5ae-1a04bcea55cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:32.553165 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e57673-dd5e-40f8-a5ae-1a04bcea55cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:32.553450 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e57673-dd5e-40f8-a5ae-1a04bcea55cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:32.553731 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e57673-dd5e-40f8-a5ae-1a04bcea55cd admin admin] PATH_INFO: `/v3/users/1ed7d867aed34eeda1dd1737269c329d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:32.554921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e57673-dd5e-40f8-a5ae-1a04bcea55cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ed7d867aed34eeda1dd1737269c329d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:32.556895 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e57673-dd5e-40f8-a5ae-1a04bcea55cd admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:32.606836 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-86e57673-dd5e-40f8-a5ae-1a04bcea55cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ed7d867aed34eeda1dd1737269c329d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:32.607609 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-86e57673-dd5e-40f8-a5ae-1a04bcea55cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ed7d867aed34eeda1dd1737269c329d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:32.608219 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-86e57673-dd5e-40f8-a5ae-1a04bcea55cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ed7d867aed34eeda1dd1737269c329d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:32.615128 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-86e57673-dd5e-40f8-a5ae-1a04bcea55cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ed7d867aed34eeda1dd1737269c329d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:32.627590 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-86e57673-dd5e-40f8-a5ae-1a04bcea55cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ed7d867aed34eeda1dd1737269c329d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:32.629251 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:28:32.641637 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1311/2629] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:28:32 2022] DELETE /identity/v3/users/1ed7d867aed34eeda1dd1737269c329d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:28:32.911372 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152c769-e846-457a-ab08-c0029a41c954 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:32.912783 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7152c769-e846-457a-ab08-c0029a41c954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:32.931172 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152c769-e846-457a-ab08-c0029a41c954 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:32.936120 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:28:32.992341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d917c-ee0d-4f16-ac2d-849b0573c046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:32.993117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d917c-ee0d-4f16-ac2d-849b0573c046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:32.993394 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d917c-ee0d-4f16-ac2d-849b0573c046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:32.993664 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d917c-ee0d-4f16-ac2d-849b0573c046 admin admin] PATH_INFO: `/v3/projects/1459c1065a0c452b8ee269410c89104b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:32.995060 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d917c-ee0d-4f16-ac2d-849b0573c046 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1459c1065a0c452b8ee269410c89104b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:32.997144 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d917c-ee0d-4f16-ac2d-849b0573c046 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:33.044496 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0b8d917c-ee0d-4f16-ac2d-849b0573c046 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1459c1065a0c452b8ee269410c89104b', 'request_id': 'req-0b8d917c-ee0d-4f16-ac2d-849b0573c046'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:33.051830 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0b8d917c-ee0d-4f16-ac2d-849b0573c046 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1459c1065a0c452b8ee269410c89104b', 'request_id': 'req-0b8d917c-ee0d-4f16-ac2d-849b0573c046'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:33.058178 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0b8d917c-ee0d-4f16-ac2d-849b0573c046 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1459c1065a0c452b8ee269410c89104b', 'request_id': 'req-0b8d917c-ee0d-4f16-ac2d-849b0573c046'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:33.068447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0b8d917c-ee0d-4f16-ac2d-849b0573c046 admin admin] The token cache is being invalidate because project 1459c1065a0c452b8ee269410c89104b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:28:33.068729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0b8d917c-ee0d-4f16-ac2d-849b0573c046 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:33.069349 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0b8d917c-ee0d-4f16-ac2d-849b0573c046 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:33.070870 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1319/2630] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:28:32 2022] DELETE /identity/v3/projects/1459c1065a0c452b8ee269410c89104b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:28:33.077648 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e2cc7-a7c1-43d0-9ada-724c3edb8c21 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:33.077947 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e2cc7-a7c1-43d0-9ada-724c3edb8c21 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:33.078254 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e2cc7-a7c1-43d0-9ada-724c3edb8c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:33.097592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0b1e2cc7-a7c1-43d0-9ada-724c3edb8c21 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:28:33.098839 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0b1e2cc7-a7c1-43d0-9ada-724c3edb8c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:33.157879 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1312/2631] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:28:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:28:33.166427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d02f45-d319-469c-9063-bb2d5483c2fe None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:33.167269 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d02f45-d319-469c-9063-bb2d5483c2fe None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:33.167543 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d02f45-d319-469c-9063-bb2d5483c2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:33.186200 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-10d02f45-d319-469c-9063-bb2d5483c2fe None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:28:33.187435 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-10d02f45-d319-469c-9063-bb2d5483c2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:33.196297 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1320/2632] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:28:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:28:33.201205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e2cc7-a7c1-43d0-9ada-724c3edb8c21 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:33.210780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e2cc7-a7c1-43d0-9ada-724c3edb8c21 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:33.215820 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae308f3-a29c-4eb8-8761-fc70a5fbece4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:33.216738 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae308f3-a29c-4eb8-8761-fc70a5fbece4 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:33.217017 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae308f3-a29c-4eb8-8761-fc70a5fbece4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:33.217291 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae308f3-a29c-4eb8-8761-fc70a5fbece4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:33.218014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae308f3-a29c-4eb8-8761-fc70a5fbece4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:28:33.218313 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae308f3-a29c-4eb8-8761-fc70a5fbece4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:33.219938 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae308f3-a29c-4eb8-8761-fc70a5fbece4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:33.231858 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1313/2633] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:28:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:28:33.236252 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d02f45-d319-469c-9063-bb2d5483c2fe None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:33.246486 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d02f45-d319-469c-9063-bb2d5483c2fe None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:33.251182 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef39b1-f1c3-4ec2-b504-a1b70d9eae26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:33.251865 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef39b1-f1c3-4ec2-b504-a1b70d9eae26 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:33.252168 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef39b1-f1c3-4ec2-b504-a1b70d9eae26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:33.252437 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef39b1-f1c3-4ec2-b504-a1b70d9eae26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:33.253115 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef39b1-f1c3-4ec2-b504-a1b70d9eae26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:33.255028 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef39b1-f1c3-4ec2-b504-a1b70d9eae26 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:33.280028 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1321/2634] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:28:33 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:28:33.284146 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae308f3-a29c-4eb8-8761-fc70a5fbece4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:33.294879 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae308f3-a29c-4eb8-8761-fc70a5fbece4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:33.350694 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88346888-cff1-4175-b335-9349e299d812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:33.351385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88346888-cff1-4175-b335-9349e299d812 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:33.351667 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88346888-cff1-4175-b335-9349e299d812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:33.351939 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88346888-cff1-4175-b335-9349e299d812 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:33.352617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88346888-cff1-4175-b335-9349e299d812 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:33.355275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88346888-cff1-4175-b335-9349e299d812 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:33.376207 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1314/2635] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:28:33 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:28:33.380401 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef39b1-f1c3-4ec2-b504-a1b70d9eae26 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:33.390938 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef39b1-f1c3-4ec2-b504-a1b70d9eae26 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:33.395620 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796171c-96db-4697-9c52-119d86b20fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:33.396275 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796171c-96db-4697-9c52-119d86b20fac admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:33.396558 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796171c-96db-4697-9c52-119d86b20fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:33.396834 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796171c-96db-4697-9c52-119d86b20fac admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:33.397580 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a796171c-96db-4697-9c52-119d86b20fac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:33.399161 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a796171c-96db-4697-9c52-119d86b20fac admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:33.409893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1322/2636] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:28:33.413678 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88346888-cff1-4175-b335-9349e299d812 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:33.424332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88346888-cff1-4175-b335-9349e299d812 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:33.428853 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daadc65-3a15-4b54-ab23-172599060dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:33.429497 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daadc65-3a15-4b54-ab23-172599060dea admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:33.429780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daadc65-3a15-4b54-ab23-172599060dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:33.430049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daadc65-3a15-4b54-ab23-172599060dea admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:33.430767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daadc65-3a15-4b54-ab23-172599060dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:33.432754 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daadc65-3a15-4b54-ab23-172599060dea admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:33.443083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1315/2637] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:28:33.447210 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796171c-96db-4697-9c52-119d86b20fac admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:33.457283 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796171c-96db-4697-9c52-119d86b20fac admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:33.462073 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f50cda-01f8-4c54-b1cd-a11f53c1fae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:33.462967 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f50cda-01f8-4c54-b1cd-a11f53c1fae1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:33.463805 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f50cda-01f8-4c54-b1cd-a11f53c1fae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:33.464449 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f50cda-01f8-4c54-b1cd-a11f53c1fae1 admin admin] PATH_INFO: `/v3/projects/0df46610e38544bbabd64837c474ae48/users/4694971aaaae4c94bc71ff159527e6bd/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:33.479253 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f50cda-01f8-4c54-b1cd-a11f53c1fae1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0df46610e38544bbabd64837c474ae48, user_id=4694971aaaae4c94bc71ff159527e6bd, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:33.481882 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f50cda-01f8-4c54-b1cd-a11f53c1fae1 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:33.494241 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1323/2638] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:28:33 2022] PUT /identity/v3/projects/0df46610e38544bbabd64837c474ae48/users/4694971aaaae4c94bc71ff159527e6bd/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:28:33.505938 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994742b-0ebe-4ea1-b4aa-15b315c8c08d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:33.506450 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994742b-0ebe-4ea1-b4aa-15b315c8c08d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:33.506450 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994742b-0ebe-4ea1-b4aa-15b315c8c08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:33.534001 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5994742b-0ebe-4ea1-b4aa-15b315c8c08d None None] MFA Rules not processed for user `4694971aaaae4c94bc71ff159527e6bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:28:33.535236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5994742b-0ebe-4ea1-b4aa-15b315c8c08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:33.578981 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:28:33.600542 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1316/2639] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:28:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:28:33.605087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f50cda-01f8-4c54-b1cd-a11f53c1fae1 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:33.640863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:28:33.669063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f50cda-01f8-4c54-b1cd-a11f53c1fae1 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:33.673696 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7734d0-62e3-489d-9cb3-78b358f6a4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:33.674359 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7734d0-62e3-489d-9cb3-78b358f6a4f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:33.674643 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7734d0-62e3-489d-9cb3-78b358f6a4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:33.674918 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7734d0-62e3-489d-9cb3-78b358f6a4f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:33.675630 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7734d0-62e3-489d-9cb3-78b358f6a4f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:33.677518 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7734d0-62e3-489d-9cb3-78b358f6a4f5 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:33.704337 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1324/2640] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:28:33 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:28:33.708119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994742b-0ebe-4ea1-b4aa-15b315c8c08d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:33.720127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994742b-0ebe-4ea1-b4aa-15b315c8c08d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:33.774778 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535e198-bcf0-4165-9304-6b0bc68681a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:33.775408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535e198-bcf0-4165-9304-6b0bc68681a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:33.775693 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535e198-bcf0-4165-9304-6b0bc68681a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:33.776671 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535e198-bcf0-4165-9304-6b0bc68681a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:33.777365 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535e198-bcf0-4165-9304-6b0bc68681a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:33.779282 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535e198-bcf0-4165-9304-6b0bc68681a2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:33.798395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1317/2641] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:28:33 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:28:33.802667 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7734d0-62e3-489d-9cb3-78b358f6a4f5 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:33.813514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7734d0-62e3-489d-9cb3-78b358f6a4f5 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:33.818179 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15955ce-73e0-4743-9261-09417819916a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:33.818843 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15955ce-73e0-4743-9261-09417819916a admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:33.819126 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15955ce-73e0-4743-9261-09417819916a admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:33.819530 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15955ce-73e0-4743-9261-09417819916a admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:33.820288 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15955ce-73e0-4743-9261-09417819916a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:33.821881 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15955ce-73e0-4743-9261-09417819916a admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:33.832859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1325/2642] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:28:33.837028 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535e198-bcf0-4165-9304-6b0bc68681a2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:33.847992 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535e198-bcf0-4165-9304-6b0bc68681a2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:33.852586 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d2e35-f013-4aa0-9b35-68c716788438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:33.853492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d2e35-f013-4aa0-9b35-68c716788438 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:33.853776 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d2e35-f013-4aa0-9b35-68c716788438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:33.854122 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d2e35-f013-4aa0-9b35-68c716788438 admin admin] PATH_INFO: `/v3/projects/8d1e8776e6b849c1bd86cf05fc5804c4/users/24bf2dcb0b18496a8a0f9484e9b30330/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:33.868480 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d2e35-f013-4aa0-9b35-68c716788438 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d1e8776e6b849c1bd86cf05fc5804c4, user_id=24bf2dcb0b18496a8a0f9484e9b30330, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:33.871166 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d2e35-f013-4aa0-9b35-68c716788438 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:33.884292 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1318/2643] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:28:33 2022] PUT /identity/v3/projects/8d1e8776e6b849c1bd86cf05fc5804c4/users/24bf2dcb0b18496a8a0f9484e9b30330/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:28:33.888807 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15955ce-73e0-4743-9261-09417819916a admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:33.898779 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15955ce-73e0-4743-9261-09417819916a admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:33.954951 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d090a8d-7d51-4c0f-98c2-b7b0befa701d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:33.955653 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d090a8d-7d51-4c0f-98c2-b7b0befa701d admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:33.955940 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d090a8d-7d51-4c0f-98c2-b7b0befa701d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:33.956219 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d090a8d-7d51-4c0f-98c2-b7b0befa701d admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:33.956945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d090a8d-7d51-4c0f-98c2-b7b0befa701d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:33.958578 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d090a8d-7d51-4c0f-98c2-b7b0befa701d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:33.969396 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1326/2644] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:28:33.973667 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d2e35-f013-4aa0-9b35-68c716788438 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:33.983810 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d2e35-f013-4aa0-9b35-68c716788438 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:33.988652 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf50522-9469-4e63-a409-16cafa1486e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:33.989523 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf50522-9469-4e63-a409-16cafa1486e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:33.989803 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf50522-9469-4e63-a409-16cafa1486e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:33.990074 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf50522-9469-4e63-a409-16cafa1486e4 admin admin] PATH_INFO: `/v3/projects/8d1e8776e6b849c1bd86cf05fc5804c4/users/24bf2dcb0b18496a8a0f9484e9b30330/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:33.992050 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf50522-9469-4e63-a409-16cafa1486e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d1e8776e6b849c1bd86cf05fc5804c4, user_id=24bf2dcb0b18496a8a0f9484e9b30330, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:33.995069 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf50522-9469-4e63-a409-16cafa1486e4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:34.005602 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1319/2645] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:28:33 2022] PUT /identity/v3/projects/8d1e8776e6b849c1bd86cf05fc5804c4/users/24bf2dcb0b18496a8a0f9484e9b30330/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:28:34.017403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895bc34-ed4e-426d-97b7-3ee388bd6bf6 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:34.017691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895bc34-ed4e-426d-97b7-3ee388bd6bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:34.017968 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895bc34-ed4e-426d-97b7-3ee388bd6bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:34.045255 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-b895bc34-ed4e-426d-97b7-3ee388bd6bf6 None None] MFA Rules not processed for user `24bf2dcb0b18496a8a0f9484e9b30330`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:28:34.046390 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b895bc34-ed4e-426d-97b7-3ee388bd6bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:34.113869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1327/2646] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:28:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:28:34.125516 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf50522-9469-4e63-a409-16cafa1486e4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:34.127244 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-aaf50522-9469-4e63-a409-16cafa1486e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:34.185155 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf50522-9469-4e63-a409-16cafa1486e4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:34.188860 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08589e-97ec-42a5-ac72-f4bc6d88c840 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:34.189511 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08589e-97ec-42a5-ac72-f4bc6d88c840 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:34.189826 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08589e-97ec-42a5-ac72-f4bc6d88c840 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:34.190101 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08589e-97ec-42a5-ac72-f4bc6d88c840 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:34.199121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08589e-97ec-42a5-ac72-f4bc6d88c840 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:34.200573 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08589e-97ec-42a5-ac72-f4bc6d88c840 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:34.209941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1320/2647] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:28:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:28:34.568271 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895bc34-ed4e-426d-97b7-3ee388bd6bf6 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:34.601433 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:28:34.634768 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895bc34-ed4e-426d-97b7-3ee388bd6bf6 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:34.639391 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379a822-3fec-45a1-b22f-f4d77e783b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:34.640261 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379a822-3fec-45a1-b22f-f4d77e783b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:34.640544 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379a822-3fec-45a1-b22f-f4d77e783b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:34.640822 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379a822-3fec-45a1-b22f-f4d77e783b2e admin admin] PATH_INFO: `/v3/users/4694971aaaae4c94bc71ff159527e6bd` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:34.642005 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379a822-3fec-45a1-b22f-f4d77e783b2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4694971aaaae4c94bc71ff159527e6bd)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:34.643957 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379a822-3fec-45a1-b22f-f4d77e783b2e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:34.689943 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4379a822-3fec-45a1-b22f-f4d77e783b2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4694971aaaae4c94bc71ff159527e6bd'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:34.701919 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4379a822-3fec-45a1-b22f-f4d77e783b2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4694971aaaae4c94bc71ff159527e6bd'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:34.702582 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4379a822-3fec-45a1-b22f-f4d77e783b2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4694971aaaae4c94bc71ff159527e6bd'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:34.703190 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4379a822-3fec-45a1-b22f-f4d77e783b2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4694971aaaae4c94bc71ff159527e6bd'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:34.709964 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4379a822-3fec-45a1-b22f-f4d77e783b2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4694971aaaae4c94bc71ff159527e6bd'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:34.722500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1328/2648] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:28:34 2022] DELETE /identity/v3/users/4694971aaaae4c94bc71ff159527e6bd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:28:34.727030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cac82f-55ec-451d-b0c6-ff40f76026a7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:34.727540 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cac82f-55ec-451d-b0c6-ff40f76026a7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:34.727681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cac82f-55ec-451d-b0c6-ff40f76026a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:34.746197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-12cac82f-55ec-451d-b0c6-ff40f76026a7 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:28:34.747432 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-12cac82f-55ec-451d-b0c6-ff40f76026a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:34.783918 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:28:34.805574 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1321/2649] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:28:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:28:34.816007 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379a822-3fec-45a1-b22f-f4d77e783b2e admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:34.817386 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4379a822-3fec-45a1-b22f-f4d77e783b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:34.875209 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379a822-3fec-45a1-b22f-f4d77e783b2e admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:34.878782 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17ca72-a056-4562-95fa-5fca090332ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:34.879450 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17ca72-a056-4562-95fa-5fca090332ba service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:34.879729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17ca72-a056-4562-95fa-5fca090332ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:34.880001 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17ca72-a056-4562-95fa-5fca090332ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:34.889134 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca17ca72-a056-4562-95fa-5fca090332ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:34.890574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca17ca72-a056-4562-95fa-5fca090332ba service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:34.901603 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1329/2650] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:28:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:28:35.172202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cac82f-55ec-451d-b0c6-ff40f76026a7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:35.173691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-12cac82f-55ec-451d-b0c6-ff40f76026a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:35.186089 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cac82f-55ec-451d-b0c6-ff40f76026a7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:35.190938 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e60ca8-86e6-42ee-a367-6e2d41fb9173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:35.191819 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e60ca8-86e6-42ee-a367-6e2d41fb9173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:35.192096 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e60ca8-86e6-42ee-a367-6e2d41fb9173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:35.192369 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e60ca8-86e6-42ee-a367-6e2d41fb9173 admin admin] PATH_INFO: `/v3/projects/0df46610e38544bbabd64837c474ae48` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:35.193556 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e60ca8-86e6-42ee-a367-6e2d41fb9173 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0df46610e38544bbabd64837c474ae48)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:35.195485 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e60ca8-86e6-42ee-a367-6e2d41fb9173 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:35.242944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a5e60ca8-86e6-42ee-a367-6e2d41fb9173 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0df46610e38544bbabd64837c474ae48', 'request_id': 'req-a5e60ca8-86e6-42ee-a367-6e2d41fb9173'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:35.250047 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a5e60ca8-86e6-42ee-a367-6e2d41fb9173 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0df46610e38544bbabd64837c474ae48', 'request_id': 'req-a5e60ca8-86e6-42ee-a367-6e2d41fb9173'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:35.255593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a5e60ca8-86e6-42ee-a367-6e2d41fb9173 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0df46610e38544bbabd64837c474ae48', 'request_id': 'req-a5e60ca8-86e6-42ee-a367-6e2d41fb9173'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:35.266564 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a5e60ca8-86e6-42ee-a367-6e2d41fb9173 admin admin] The token cache is being invalidate because project 0df46610e38544bbabd64837c474ae48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:28:35.266851 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a5e60ca8-86e6-42ee-a367-6e2d41fb9173 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:35.267571 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a5e60ca8-86e6-42ee-a367-6e2d41fb9173 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:35.269175 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1322/2651] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:28:35 2022] DELETE /identity/v3/projects/0df46610e38544bbabd64837c474ae48 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:28:35.273341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17ca72-a056-4562-95fa-5fca090332ba service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:35.283458 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17ca72-a056-4562-95fa-5fca090332ba service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:35.336166 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:28:35.348573 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e98330-0d2f-4108-94ab-4adfee2f2b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:35.349380 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e98330-0d2f-4108-94ab-4adfee2f2b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:35.349658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e98330-0d2f-4108-94ab-4adfee2f2b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:35.349953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e98330-0d2f-4108-94ab-4adfee2f2b33 admin admin] PATH_INFO: `/v3/users/24bf2dcb0b18496a8a0f9484e9b30330` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:35.351117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e98330-0d2f-4108-94ab-4adfee2f2b33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24bf2dcb0b18496a8a0f9484e9b30330)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:35.353042 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e98330-0d2f-4108-94ab-4adfee2f2b33 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:35.397762 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e5e98330-0d2f-4108-94ab-4adfee2f2b33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24bf2dcb0b18496a8a0f9484e9b30330'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:35.410194 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e5e98330-0d2f-4108-94ab-4adfee2f2b33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24bf2dcb0b18496a8a0f9484e9b30330'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:35.410864 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e5e98330-0d2f-4108-94ab-4adfee2f2b33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24bf2dcb0b18496a8a0f9484e9b30330'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:35.411498 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e5e98330-0d2f-4108-94ab-4adfee2f2b33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24bf2dcb0b18496a8a0f9484e9b30330'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:35.418224 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e5e98330-0d2f-4108-94ab-4adfee2f2b33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24bf2dcb0b18496a8a0f9484e9b30330'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:35.430363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1330/2652] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:28:35 2022] DELETE /identity/v3/users/24bf2dcb0b18496a8a0f9484e9b30330 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:28:35.702050 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e60ca8-86e6-42ee-a367-6e2d41fb9173 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:35.703572 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a5e60ca8-86e6-42ee-a367-6e2d41fb9173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:35.717558 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e60ca8-86e6-42ee-a367-6e2d41fb9173 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:35.774255 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9550adb8-1693-4f31-b5c5-a2e2a3307819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:35.775101 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550adb8-1693-4f31-b5c5-a2e2a3307819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:35.775415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550adb8-1693-4f31-b5c5-a2e2a3307819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:35.776330 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550adb8-1693-4f31-b5c5-a2e2a3307819 admin admin] PATH_INFO: `/v3/projects/8d1e8776e6b849c1bd86cf05fc5804c4` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:35.777539 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9550adb8-1693-4f31-b5c5-a2e2a3307819 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d1e8776e6b849c1bd86cf05fc5804c4)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:35.779456 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9550adb8-1693-4f31-b5c5-a2e2a3307819 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:35.825010 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9550adb8-1693-4f31-b5c5-a2e2a3307819 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d1e8776e6b849c1bd86cf05fc5804c4', 'request_id': 'req-9550adb8-1693-4f31-b5c5-a2e2a3307819'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:35.831906 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9550adb8-1693-4f31-b5c5-a2e2a3307819 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d1e8776e6b849c1bd86cf05fc5804c4', 'request_id': 'req-9550adb8-1693-4f31-b5c5-a2e2a3307819'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:35.837294 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9550adb8-1693-4f31-b5c5-a2e2a3307819 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d1e8776e6b849c1bd86cf05fc5804c4', 'request_id': 'req-9550adb8-1693-4f31-b5c5-a2e2a3307819'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:35.840427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:28:35.856230 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9550adb8-1693-4f31-b5c5-a2e2a3307819 admin admin] The token cache is being invalidate because project 8d1e8776e6b849c1bd86cf05fc5804c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:28:35.856774 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9550adb8-1693-4f31-b5c5-a2e2a3307819 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:35.857569 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9550adb8-1693-4f31-b5c5-a2e2a3307819 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:35.859147 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1323/2653] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:28:35 2022] DELETE /identity/v3/projects/8d1e8776e6b849c1bd86cf05fc5804c4 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:28:35.864591 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3b421-a7f3-442d-b59a-b414ff48214b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:35.865095 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3b421-a7f3-442d-b59a-b414ff48214b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:35.865198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3b421-a7f3-442d-b59a-b414ff48214b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:35.883074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-5da3b421-a7f3-442d-b59a-b414ff48214b None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:28:35.884235 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5da3b421-a7f3-442d-b59a-b414ff48214b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:35.943201 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1331/2654] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:28:35.951263 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3fad1-0e3f-44e1-9e82-5f393331321b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:35.951560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3fad1-0e3f-44e1-9e82-5f393331321b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:35.951839 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3fad1-0e3f-44e1-9e82-5f393331321b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:35.970879 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b7c3fad1-0e3f-44e1-9e82-5f393331321b None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:28:35.972158 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b7c3fad1-0e3f-44e1-9e82-5f393331321b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:35.980007 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1324/2655] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:28:35.984117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3b421-a7f3-442d-b59a-b414ff48214b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:35.993702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3b421-a7f3-442d-b59a-b414ff48214b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:35.998981 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37da16-f3a1-4b69-837a-3703f4d952d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:35.999702 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37da16-f3a1-4b69-837a-3703f4d952d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:35.999982 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37da16-f3a1-4b69-837a-3703f4d952d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:36.000255 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37da16-f3a1-4b69-837a-3703f4d952d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:36.001016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37da16-f3a1-4b69-837a-3703f4d952d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:28:36.001321 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37da16-f3a1-4b69-837a-3703f4d952d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:36.002936 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37da16-f3a1-4b69-837a-3703f4d952d5 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:36.015399 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1332/2656] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:28:36.019627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3fad1-0e3f-44e1-9e82-5f393331321b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:36.029088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3fad1-0e3f-44e1-9e82-5f393331321b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:36.033735 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe04420-1564-43e2-b913-0075eb1be285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:36.034385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe04420-1564-43e2-b913-0075eb1be285 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:36.034669 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe04420-1564-43e2-b913-0075eb1be285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:36.034945 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe04420-1564-43e2-b913-0075eb1be285 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:36.035681 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe04420-1564-43e2-b913-0075eb1be285 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:36.037560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe04420-1564-43e2-b913-0075eb1be285 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:36.066245 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1325/2657] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:28:36 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:28:36.067418 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37da16-f3a1-4b69-837a-3703f4d952d5 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:36.078185 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37da16-f3a1-4b69-837a-3703f4d952d5 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:36.128236 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:28:36.135953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86d393-d3dc-4cb3-8398-f085892687da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:36.136728 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86d393-d3dc-4cb3-8398-f085892687da admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:36.137207 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86d393-d3dc-4cb3-8398-f085892687da admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:36.137781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86d393-d3dc-4cb3-8398-f085892687da admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:36.138656 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86d393-d3dc-4cb3-8398-f085892687da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:36.140901 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86d393-d3dc-4cb3-8398-f085892687da admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:36.164617 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1333/2658] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:28:36 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:28:36.166033 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe04420-1564-43e2-b913-0075eb1be285 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:36.176751 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe04420-1564-43e2-b913-0075eb1be285 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:36.181378 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51912bcd-2384-437c-8ba5-cddf08e7be1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:36.182149 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51912bcd-2384-437c-8ba5-cddf08e7be1b admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:36.182605 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51912bcd-2384-437c-8ba5-cddf08e7be1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:36.183015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51912bcd-2384-437c-8ba5-cddf08e7be1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:36.183802 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51912bcd-2384-437c-8ba5-cddf08e7be1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:36.185376 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51912bcd-2384-437c-8ba5-cddf08e7be1b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:36.195819 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1326/2659] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:28:36.200761 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86d393-d3dc-4cb3-8398-f085892687da admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:36.210497 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86d393-d3dc-4cb3-8398-f085892687da admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:36.215713 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752509d-833c-4bde-95a6-d16fc923cc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:36.216613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752509d-833c-4bde-95a6-d16fc923cc6b admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:36.216891 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752509d-833c-4bde-95a6-d16fc923cc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:36.217161 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752509d-833c-4bde-95a6-d16fc923cc6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:36.217910 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e752509d-833c-4bde-95a6-d16fc923cc6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:36.219512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e752509d-833c-4bde-95a6-d16fc923cc6b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:36.229465 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1334/2660] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:28:36.233555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51912bcd-2384-437c-8ba5-cddf08e7be1b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:36.243468 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51912bcd-2384-437c-8ba5-cddf08e7be1b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:36.248198 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ea64b-ecfa-49f4-907f-0e6ca2f3aa24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:36.249070 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ea64b-ecfa-49f4-907f-0e6ca2f3aa24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:36.249356 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ea64b-ecfa-49f4-907f-0e6ca2f3aa24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:36.249629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ea64b-ecfa-49f4-907f-0e6ca2f3aa24 admin admin] PATH_INFO: `/v3/projects/bdb4a96c50f84e049c474f783dcb9f09/users/59e9e2c11b6c4f13aa028e7767f1b439/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:36.265657 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ea64b-ecfa-49f4-907f-0e6ca2f3aa24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdb4a96c50f84e049c474f783dcb9f09, user_id=59e9e2c11b6c4f13aa028e7767f1b439, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:36.268336 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ea64b-ecfa-49f4-907f-0e6ca2f3aa24 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:36.281309 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1327/2661] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:28:36 2022] PUT /identity/v3/projects/bdb4a96c50f84e049c474f783dcb9f09/users/59e9e2c11b6c4f13aa028e7767f1b439/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:28:36.293402 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf30d0-95a6-4062-b5ed-fe0c0d5eb83f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:36.293700 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf30d0-95a6-4062-b5ed-fe0c0d5eb83f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:36.293984 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf30d0-95a6-4062-b5ed-fe0c0d5eb83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:36.320670 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-41bf30d0-95a6-4062-b5ed-fe0c0d5eb83f None None] MFA Rules not processed for user `59e9e2c11b6c4f13aa028e7767f1b439`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:28:36.321879 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-41bf30d0-95a6-4062-b5ed-fe0c0d5eb83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:36.389077 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1335/2662] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:28:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:28:36.399895 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ea64b-ecfa-49f4-907f-0e6ca2f3aa24 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:36.402321 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ce8ea64b-ecfa-49f4-907f-0e6ca2f3aa24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:36.458517 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ea64b-ecfa-49f4-907f-0e6ca2f3aa24 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:36.462131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdfa5f9-39fd-4db0-af93-c4db674a33dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:36.462775 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfa5f9-39fd-4db0-af93-c4db674a33dd service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:36.463059 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfa5f9-39fd-4db0-af93-c4db674a33dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:36.463416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfa5f9-39fd-4db0-af93-c4db674a33dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:36.471963 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdfa5f9-39fd-4db0-af93-c4db674a33dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:36.473414 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdfa5f9-39fd-4db0-af93-c4db674a33dd service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:36.482051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1328/2663] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:28:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:28:36.502913 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf30d0-95a6-4062-b5ed-fe0c0d5eb83f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:36.564794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf30d0-95a6-4062-b5ed-fe0c0d5eb83f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:36.569369 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf851473-e052-445d-aa9e-514b675af002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:36.570239 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf851473-e052-445d-aa9e-514b675af002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:36.570523 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf851473-e052-445d-aa9e-514b675af002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:36.570797 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf851473-e052-445d-aa9e-514b675af002 admin admin] PATH_INFO: `/v3/users/59e9e2c11b6c4f13aa028e7767f1b439` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:36.572019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf851473-e052-445d-aa9e-514b675af002 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59e9e2c11b6c4f13aa028e7767f1b439)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:36.574743 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf851473-e052-445d-aa9e-514b675af002 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:36.621554 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cf851473-e052-445d-aa9e-514b675af002 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59e9e2c11b6c4f13aa028e7767f1b439'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:36.633330 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cf851473-e052-445d-aa9e-514b675af002 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59e9e2c11b6c4f13aa028e7767f1b439'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:36.634008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cf851473-e052-445d-aa9e-514b675af002 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59e9e2c11b6c4f13aa028e7767f1b439'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:36.634625 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cf851473-e052-445d-aa9e-514b675af002 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59e9e2c11b6c4f13aa028e7767f1b439'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:36.636853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:28:36.642586 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cf851473-e052-445d-aa9e-514b675af002 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59e9e2c11b6c4f13aa028e7767f1b439'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:36.656065 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1336/2664] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:28:36 2022] DELETE /identity/v3/users/59e9e2c11b6c4f13aa028e7767f1b439 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:28:36.660828 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf36f5-ec92-4a90-a4a0-478ca768891f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:36.661125 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf36f5-ec92-4a90-a4a0-478ca768891f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:36.661404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf36f5-ec92-4a90-a4a0-478ca768891f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:36.678959 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ccdf36f5-ec92-4a90-a4a0-478ca768891f None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:28:36.680195 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ccdf36f5-ec92-4a90-a4a0-478ca768891f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:36.738516 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1329/2665] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:28:36.748222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf851473-e052-445d-aa9e-514b675af002 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:36.749682 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-cf851473-e052-445d-aa9e-514b675af002 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:36.806157 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf851473-e052-445d-aa9e-514b675af002 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:36.809978 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bec052-15ca-460e-9ac7-ce3cdf658bb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:36.810618 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bec052-15ca-460e-9ac7-ce3cdf658bb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:36.810898 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bec052-15ca-460e-9ac7-ce3cdf658bb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:36.811172 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bec052-15ca-460e-9ac7-ce3cdf658bb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:36.819940 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bec052-15ca-460e-9ac7-ce3cdf658bb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:36.821403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bec052-15ca-460e-9ac7-ce3cdf658bb7 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:36.831526 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1337/2666] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:28:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:28:37.096790 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf36f5-ec92-4a90-a4a0-478ca768891f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:37.098224 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ccdf36f5-ec92-4a90-a4a0-478ca768891f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:37.110795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf36f5-ec92-4a90-a4a0-478ca768891f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:37.115893 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d813a4-1a85-4cee-ba0f-67a00928735d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:37.116763 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d813a4-1a85-4cee-ba0f-67a00928735d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:37.117048 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d813a4-1a85-4cee-ba0f-67a00928735d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:37.117324 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d813a4-1a85-4cee-ba0f-67a00928735d admin admin] PATH_INFO: `/v3/projects/bdb4a96c50f84e049c474f783dcb9f09` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:37.118541 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d813a4-1a85-4cee-ba0f-67a00928735d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdb4a96c50f84e049c474f783dcb9f09)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:37.121281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d813a4-1a85-4cee-ba0f-67a00928735d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:37.155409 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:28:37.170402 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-66d813a4-1a85-4cee-ba0f-67a00928735d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdb4a96c50f84e049c474f783dcb9f09', 'request_id': 'req-66d813a4-1a85-4cee-ba0f-67a00928735d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:37.177900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-66d813a4-1a85-4cee-ba0f-67a00928735d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdb4a96c50f84e049c474f783dcb9f09', 'request_id': 'req-66d813a4-1a85-4cee-ba0f-67a00928735d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:37.184045 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-66d813a4-1a85-4cee-ba0f-67a00928735d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdb4a96c50f84e049c474f783dcb9f09', 'request_id': 'req-66d813a4-1a85-4cee-ba0f-67a00928735d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:37.194931 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-66d813a4-1a85-4cee-ba0f-67a00928735d admin admin] The token cache is being invalidate because project bdb4a96c50f84e049c474f783dcb9f09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:28:37.195236 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-66d813a4-1a85-4cee-ba0f-67a00928735d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:37.195880 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-66d813a4-1a85-4cee-ba0f-67a00928735d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:28:37.197466 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1330/2667] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:28:37 2022] DELETE /identity/v3/projects/bdb4a96c50f84e049c474f783dcb9f09 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:28:37.203777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87171e82-259b-41d9-a631-1376236c2a6e None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:37.204257 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87171e82-259b-41d9-a631-1376236c2a6e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:37.204377 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87171e82-259b-41d9-a631-1376236c2a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:37.223629 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-87171e82-259b-41d9-a631-1376236c2a6e None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:28:37.224812 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-87171e82-259b-41d9-a631-1376236c2a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:37.283823 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1338/2668] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:28:37.291682 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575f9bb-81de-4485-a89c-26f3947cd6e9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:37.292196 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575f9bb-81de-4485-a89c-26f3947cd6e9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:37.292508 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575f9bb-81de-4485-a89c-26f3947cd6e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:37.310850 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c575f9bb-81de-4485-a89c-26f3947cd6e9 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:28:37.312068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c575f9bb-81de-4485-a89c-26f3947cd6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:37.319763 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1331/2669] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:28:37.324794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87171e82-259b-41d9-a631-1376236c2a6e None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:37.334414 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87171e82-259b-41d9-a631-1376236c2a6e None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:37.339137 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a710f-3402-42b2-9599-649020fb06ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:37.339826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a710f-3402-42b2-9599-649020fb06ed admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:37.340114 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a710f-3402-42b2-9599-649020fb06ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:37.340399 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a710f-3402-42b2-9599-649020fb06ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:37.341390 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a710f-3402-42b2-9599-649020fb06ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:28:37.341712 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a710f-3402-42b2-9599-649020fb06ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:37.343628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a710f-3402-42b2-9599-649020fb06ed admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:37.355599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1339/2670] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:28:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:28:37.359756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c575f9bb-81de-4485-a89c-26f3947cd6e9 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:37.368787 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c575f9bb-81de-4485-a89c-26f3947cd6e9 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:37.374214 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e28820-8899-49ca-af6c-2e0667580786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:37.374864 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e28820-8899-49ca-af6c-2e0667580786 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:37.375144 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e28820-8899-49ca-af6c-2e0667580786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:37.375447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e28820-8899-49ca-af6c-2e0667580786 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:37.376138 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e28820-8899-49ca-af6c-2e0667580786 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:37.378011 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e28820-8899-49ca-af6c-2e0667580786 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:37.404190 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1332/2671] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:28:37 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:28:37.408258 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a710f-3402-42b2-9599-649020fb06ed admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:37.418263 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a710f-3402-42b2-9599-649020fb06ed admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:37.474993 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d5bab-6f98-41c0-a1d5-dad3b184ce08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:37.475696 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d5bab-6f98-41c0-a1d5-dad3b184ce08 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:37.475977 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d5bab-6f98-41c0-a1d5-dad3b184ce08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:37.476252 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d5bab-6f98-41c0-a1d5-dad3b184ce08 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:37.476940 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d5bab-6f98-41c0-a1d5-dad3b184ce08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:37.478933 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d5bab-6f98-41c0-a1d5-dad3b184ce08 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:37.498179 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1340/2672] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:28:37 2022] POST /identity/v3/users => generated 283 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:28:37.502313 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e28820-8899-49ca-af6c-2e0667580786 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:37.512980 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e28820-8899-49ca-af6c-2e0667580786 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:37.517498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4b99b-7bf9-417c-9328-8af9805171af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:37.518148 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4b99b-7bf9-417c-9328-8af9805171af admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:37.518429 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4b99b-7bf9-417c-9328-8af9805171af admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:37.518702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4b99b-7bf9-417c-9328-8af9805171af admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:37.519463 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4b99b-7bf9-417c-9328-8af9805171af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:37.521058 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4b99b-7bf9-417c-9328-8af9805171af admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:37.531185 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1333/2673] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:28:37.534986 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d5bab-6f98-41c0-a1d5-dad3b184ce08 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:37.542519 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:28:37.546245 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d5bab-6f98-41c0-a1d5-dad3b184ce08 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:37.552250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe36d6-ebec-4b4b-957c-92f41f3fc55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:37.553101 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe36d6-ebec-4b4b-957c-92f41f3fc55d admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:37.553570 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe36d6-ebec-4b4b-957c-92f41f3fc55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:37.553887 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe36d6-ebec-4b4b-957c-92f41f3fc55d admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:37.554792 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbe36d6-ebec-4b4b-957c-92f41f3fc55d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:37.556609 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbe36d6-ebec-4b4b-957c-92f41f3fc55d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:37.567035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1341/2674] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:28:37.571006 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4b99b-7bf9-417c-9328-8af9805171af admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:37.580911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4b99b-7bf9-417c-9328-8af9805171af admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:37.585525 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34978f67-40f4-4f84-b7e5-695c73cc2035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:37.586423 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34978f67-40f4-4f84-b7e5-695c73cc2035 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:37.586979 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34978f67-40f4-4f84-b7e5-695c73cc2035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:37.586979 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34978f67-40f4-4f84-b7e5-695c73cc2035 admin admin] PATH_INFO: `/v3/projects/dd407a635267491eb6a4bea70ffc8fd1/users/5afa67432dfb4825b224d9158b5a8dfe/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:37.602244 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34978f67-40f4-4f84-b7e5-695c73cc2035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd407a635267491eb6a4bea70ffc8fd1, user_id=5afa67432dfb4825b224d9158b5a8dfe, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:37.605990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34978f67-40f4-4f84-b7e5-695c73cc2035 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:37.618499 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1334/2675] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:28:37 2022] PUT /identity/v3/projects/dd407a635267491eb6a4bea70ffc8fd1/users/5afa67432dfb4825b224d9158b5a8dfe/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:28:37.629400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646decf-d227-4751-8f68-47fa54437b3c None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:37.629712 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646decf-d227-4751-8f68-47fa54437b3c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:37.629989 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646decf-d227-4751-8f68-47fa54437b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:37.656103 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a646decf-d227-4751-8f68-47fa54437b3c None None] MFA Rules not processed for user `5afa67432dfb4825b224d9158b5a8dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:28:37.657306 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a646decf-d227-4751-8f68-47fa54437b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:37.720124 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1342/2676] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:28:37 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:28:37.731869 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34978f67-40f4-4f84-b7e5-695c73cc2035 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:28:37.733559 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-34978f67-40f4-4f84-b7e5-695c73cc2035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:28:37.789884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34978f67-40f4-4f84-b7e5-695c73cc2035 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:28:37.794257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d08b8-dd9f-4d0c-9a44-3c245be63aa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:28:37.794904 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d08b8-dd9f-4d0c-9a44-3c245be63aa6 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:28:37.795188 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d08b8-dd9f-4d0c-9a44-3c245be63aa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:28:37.795568 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d08b8-dd9f-4d0c-9a44-3c245be63aa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:28:37.803928 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d08b8-dd9f-4d0c-9a44-3c245be63aa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:28:37.805362 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d08b8-dd9f-4d0c-9a44-3c245be63aa6 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:28:37.815341 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1335/2677] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:28:37 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:28:47.717079 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=35 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:28:47.811421 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=66,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:29:05.852659 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a646decf-d227-4751-8f68-47fa54437b3c None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:05.854074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a646decf-d227-4751-8f68-47fa54437b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:05.911071 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a646decf-d227-4751-8f68-47fa54437b3c None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:05.914640 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14bceb9-1759-43d0-8f01-dd79b4eacceb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:05.915312 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bceb9-1759-43d0-8f01-dd79b4eacceb service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:05.915598 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bceb9-1759-43d0-8f01-dd79b4eacceb service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:05.915871 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bceb9-1759-43d0-8f01-dd79b4eacceb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:05.917695 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f14bceb9-1759-43d0-8f01-dd79b4eacceb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:05.976669 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14bceb9-1759-43d0-8f01-dd79b4eacceb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:05.978101 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14bceb9-1759-43d0-8f01-dd79b4eacceb service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:05.987857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1343/2678] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:29:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:29:10.710121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d08b8-dd9f-4d0c-9a44-3c245be63aa6 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:10.720178 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d08b8-dd9f-4d0c-9a44-3c245be63aa6 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:10.723915 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c60e8c-4a84-43c4-8c1a-e30761f8eb79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:10.726909 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c60e8c-4a84-43c4-8c1a-e30761f8eb79 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:10.727574 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c60e8c-4a84-43c4-8c1a-e30761f8eb79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:10.727899 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c60e8c-4a84-43c4-8c1a-e30761f8eb79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:10.729765 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f7c60e8c-4a84-43c4-8c1a-e30761f8eb79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:10.793273 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c60e8c-4a84-43c4-8c1a-e30761f8eb79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:10.794701 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c60e8c-4a84-43c4-8c1a-e30761f8eb79 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:10.805107 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1336/2679] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:29:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:29:12.646742 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14bceb9-1759-43d0-8f01-dd79b4eacceb service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:12.658924 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14bceb9-1759-43d0-8f01-dd79b4eacceb service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:12.663799 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0388673-f669-4042-a13b-f2abef8f1c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:12.664654 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0388673-f669-4042-a13b-f2abef8f1c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:12.664935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0388673-f669-4042-a13b-f2abef8f1c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:12.665211 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0388673-f669-4042-a13b-f2abef8f1c2d admin admin] PATH_INFO: `/v3/users/5afa67432dfb4825b224d9158b5a8dfe` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:12.666579 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0388673-f669-4042-a13b-f2abef8f1c2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5afa67432dfb4825b224d9158b5a8dfe)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:12.668517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0388673-f669-4042-a13b-f2abef8f1c2d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:12.721893 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d0388673-f669-4042-a13b-f2abef8f1c2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5afa67432dfb4825b224d9158b5a8dfe'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:12.744087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d0388673-f669-4042-a13b-f2abef8f1c2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5afa67432dfb4825b224d9158b5a8dfe'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:12.744962 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d0388673-f669-4042-a13b-f2abef8f1c2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5afa67432dfb4825b224d9158b5a8dfe'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:12.746175 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d0388673-f669-4042-a13b-f2abef8f1c2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5afa67432dfb4825b224d9158b5a8dfe'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:12.754824 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d0388673-f669-4042-a13b-f2abef8f1c2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5afa67432dfb4825b224d9158b5a8dfe'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:12.771424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1344/2680] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:29:12 2022] DELETE /identity/v3/users/5afa67432dfb4825b224d9158b5a8dfe => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:29:12.776716 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b83d4-03dc-4b7b-b23b-a274f2328308 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:12.778174 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b83d4-03dc-4b7b-b23b-a274f2328308 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:12.778801 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b83d4-03dc-4b7b-b23b-a274f2328308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:12.802955 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-034b83d4-03dc-4b7b-b23b-a274f2328308 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:29:12.804452 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-034b83d4-03dc-4b7b-b23b-a274f2328308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:12.896258 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1337/2681] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:29:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:29:12.906597 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0388673-f669-4042-a13b-f2abef8f1c2d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:12.908096 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d0388673-f669-4042-a13b-f2abef8f1c2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:12.946852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:29:12.979798 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0388673-f669-4042-a13b-f2abef8f1c2d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:12.983785 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5cba6-f401-4469-adcb-a26ae68cf348 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:12.984567 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5cba6-f401-4469-adcb-a26ae68cf348 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:12.984915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5cba6-f401-4469-adcb-a26ae68cf348 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:12.985252 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5cba6-f401-4469-adcb-a26ae68cf348 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:12.995252 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda5cba6-f401-4469-adcb-a26ae68cf348 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:12.996801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda5cba6-f401-4469-adcb-a26ae68cf348 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:13.009759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1345/2682] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:29:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:29:13.302251 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b83d4-03dc-4b7b-b23b-a274f2328308 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:13.303819 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-034b83d4-03dc-4b7b-b23b-a274f2328308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:13.330128 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b83d4-03dc-4b7b-b23b-a274f2328308 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:13.339344 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed8fda-df7b-48bc-844f-2631dc55fe13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:13.340404 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed8fda-df7b-48bc-844f-2631dc55fe13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:13.340684 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed8fda-df7b-48bc-844f-2631dc55fe13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:13.340957 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed8fda-df7b-48bc-844f-2631dc55fe13 admin admin] PATH_INFO: `/v3/projects/dd407a635267491eb6a4bea70ffc8fd1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:13.342151 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed8fda-df7b-48bc-844f-2631dc55fe13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd407a635267491eb6a4bea70ffc8fd1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:13.344640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed8fda-df7b-48bc-844f-2631dc55fe13 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:13.400904 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-51ed8fda-df7b-48bc-844f-2631dc55fe13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd407a635267491eb6a4bea70ffc8fd1', 'request_id': 'req-51ed8fda-df7b-48bc-844f-2631dc55fe13'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:13.408387 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-51ed8fda-df7b-48bc-844f-2631dc55fe13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd407a635267491eb6a4bea70ffc8fd1', 'request_id': 'req-51ed8fda-df7b-48bc-844f-2631dc55fe13'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:13.415244 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-51ed8fda-df7b-48bc-844f-2631dc55fe13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd407a635267491eb6a4bea70ffc8fd1', 'request_id': 'req-51ed8fda-df7b-48bc-844f-2631dc55fe13'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:13.428540 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-51ed8fda-df7b-48bc-844f-2631dc55fe13 admin admin] The token cache is being invalidate because project dd407a635267491eb6a4bea70ffc8fd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:29:13.428830 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-51ed8fda-df7b-48bc-844f-2631dc55fe13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:13.429511 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-51ed8fda-df7b-48bc-844f-2631dc55fe13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:13.431107 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1338/2683] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:29:13 2022] DELETE /identity/v3/projects/dd407a635267491eb6a4bea70ffc8fd1 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:29:13.435929 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae91722-064c-41c1-8d12-366e6e7f7fd4 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:13.436216 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae91722-064c-41c1-8d12-366e6e7f7fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:13.436703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae91722-064c-41c1-8d12-366e6e7f7fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:13.454863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-4ae91722-064c-41c1-8d12-366e6e7f7fd4 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:29:13.456123 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4ae91722-064c-41c1-8d12-366e6e7f7fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:13.519474 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1346/2684] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:29:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:29:13.527351 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505da8f-dcce-4d53-a953-62320de5d7b8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:13.527717 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505da8f-dcce-4d53-a953-62320de5d7b8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:13.528088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505da8f-dcce-4d53-a953-62320de5d7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:13.552519 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5505da8f-dcce-4d53-a953-62320de5d7b8 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:29:13.553821 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5505da8f-dcce-4d53-a953-62320de5d7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:13.563470 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1339/2685] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:29:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:29:13.567348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae91722-064c-41c1-8d12-366e6e7f7fd4 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:13.578001 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae91722-064c-41c1-8d12-366e6e7f7fd4 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:13.582918 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ebdf8-d4b4-47d8-ada3-b2a3c055b2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:13.583618 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ebdf8-d4b4-47d8-ada3-b2a3c055b2cb admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:13.583905 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ebdf8-d4b4-47d8-ada3-b2a3c055b2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:13.584182 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ebdf8-d4b4-47d8-ada3-b2a3c055b2cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:13.584918 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ebdf8-d4b4-47d8-ada3-b2a3c055b2cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:29:13.585228 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ebdf8-d4b4-47d8-ada3-b2a3c055b2cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:13.586820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ebdf8-d4b4-47d8-ada3-b2a3c055b2cb admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:13.600306 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1347/2686] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:29:13.604454 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505da8f-dcce-4d53-a953-62320de5d7b8 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:13.615976 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505da8f-dcce-4d53-a953-62320de5d7b8 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:13.624802 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a73987-3f65-42a2-b9d1-4ce811296a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:13.625483 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a73987-3f65-42a2-b9d1-4ce811296a1f admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:13.625770 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a73987-3f65-42a2-b9d1-4ce811296a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:13.626045 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a73987-3f65-42a2-b9d1-4ce811296a1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:13.626747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a73987-3f65-42a2-b9d1-4ce811296a1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:13.628711 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a73987-3f65-42a2-b9d1-4ce811296a1f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:13.637415 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:29:13.665611 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1340/2687] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:29:13 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:29:13.669551 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ebdf8-d4b4-47d8-ada3-b2a3c055b2cb admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:13.686259 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ebdf8-d4b4-47d8-ada3-b2a3c055b2cb admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:13.777667 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a6408-e6f4-4644-a90b-f82dc16f1ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:13.778359 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a6408-e6f4-4644-a90b-f82dc16f1ace admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:13.778644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a6408-e6f4-4644-a90b-f82dc16f1ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:13.778920 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a6408-e6f4-4644-a90b-f82dc16f1ace admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:13.779667 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a6408-e6f4-4644-a90b-f82dc16f1ace admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:13.781917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a6408-e6f4-4644-a90b-f82dc16f1ace admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:13.816328 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1348/2688] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:29:13 2022] POST /identity/v3/users => generated 288 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:29:13.819878 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a73987-3f65-42a2-b9d1-4ce811296a1f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:13.835805 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a73987-3f65-42a2-b9d1-4ce811296a1f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:13.845071 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6225c8-6518-4fab-9dd8-73a0773cf878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:13.846417 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6225c8-6518-4fab-9dd8-73a0773cf878 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:13.847035 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6225c8-6518-4fab-9dd8-73a0773cf878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:13.847531 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6225c8-6518-4fab-9dd8-73a0773cf878 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:13.848886 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6225c8-6518-4fab-9dd8-73a0773cf878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:13.851083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6225c8-6518-4fab-9dd8-73a0773cf878 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:13.866618 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1341/2689] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:29:13 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:29:13.870446 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a6408-e6f4-4644-a90b-f82dc16f1ace admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:13.883753 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a6408-e6f4-4644-a90b-f82dc16f1ace admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:13.890343 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79eab8-8880-43d5-9fde-b2f4bf93a83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:13.890799 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79eab8-8880-43d5-9fde-b2f4bf93a83f admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:13.891318 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79eab8-8880-43d5-9fde-b2f4bf93a83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:13.891655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79eab8-8880-43d5-9fde-b2f4bf93a83f admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:13.892593 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab79eab8-8880-43d5-9fde-b2f4bf93a83f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:13.896302 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab79eab8-8880-43d5-9fde-b2f4bf93a83f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:13.917436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6225c8-6518-4fab-9dd8-73a0773cf878 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:13.924066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1349/2690] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:29:13 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:29:13.929731 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6225c8-6518-4fab-9dd8-73a0773cf878 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:13.940816 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015dfd8-0210-47b4-b89d-88b9a19510c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:13.942197 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015dfd8-0210-47b4-b89d-88b9a19510c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:13.942864 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015dfd8-0210-47b4-b89d-88b9a19510c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:13.943408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015dfd8-0210-47b4-b89d-88b9a19510c4 admin admin] PATH_INFO: `/v3/projects/6e8cfd2bd8134c328696944b1913bc41/users/4185a7263943447a92a15571f192b0f8/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:13.984988 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4015dfd8-0210-47b4-b89d-88b9a19510c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e8cfd2bd8134c328696944b1913bc41, user_id=4185a7263943447a92a15571f192b0f8, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:13.996905 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4015dfd8-0210-47b4-b89d-88b9a19510c4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:14.025244 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1342/2691] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:29:13 2022] PUT /identity/v3/projects/6e8cfd2bd8134c328696944b1913bc41/users/4185a7263943447a92a15571f192b0f8/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:29:14.037310 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323315e4-68ae-4380-8b18-68c5c5a5ccc6 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:14.038044 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323315e4-68ae-4380-8b18-68c5c5a5ccc6 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:14.038464 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323315e4-68ae-4380-8b18-68c5c5a5ccc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:14.097066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-323315e4-68ae-4380-8b18-68c5c5a5ccc6 None None] MFA Rules not processed for user `4185a7263943447a92a15571f192b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:29:14.098823 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-323315e4-68ae-4380-8b18-68c5c5a5ccc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:14.145857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:29:14.200494 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1350/2692] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:29:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:29:14.211671 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015dfd8-0210-47b4-b89d-88b9a19510c4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:14.213445 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4015dfd8-0210-47b4-b89d-88b9a19510c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:14.283064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015dfd8-0210-47b4-b89d-88b9a19510c4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:14.286764 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaef025-8e15-4a78-a00c-e38f29e8b869 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:14.287445 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaef025-8e15-4a78-a00c-e38f29e8b869 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:14.287730 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaef025-8e15-4a78-a00c-e38f29e8b869 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:14.288004 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaef025-8e15-4a78-a00c-e38f29e8b869 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:14.296710 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaef025-8e15-4a78-a00c-e38f29e8b869 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:14.298140 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaef025-8e15-4a78-a00c-e38f29e8b869 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:14.307305 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1343/2693] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:29:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:29:15.663808 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7aa00-15c4-4385-b5fc-10e1bfcb6357 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:15.664294 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7aa00-15c4-4385-b5fc-10e1bfcb6357 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:15.664410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7aa00-15c4-4385-b5fc-10e1bfcb6357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:15.683435 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-90f7aa00-15c4-4385-b5fc-10e1bfcb6357 None None] MFA Rules not processed for user `4185a7263943447a92a15571f192b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:29:15.684643 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-90f7aa00-15c4-4385-b5fc-10e1bfcb6357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:15.693346 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1351/2694] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:29:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:29:18.269737 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaef025-8e15-4a78-a00c-e38f29e8b869 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:18.271617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ddaef025-8e15-4a78-a00c-e38f29e8b869 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:18.340202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaef025-8e15-4a78-a00c-e38f29e8b869 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:18.345017 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b2b1b-71e5-4d40-bf9a-97e338540f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:18.345835 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b2b1b-71e5-4d40-bf9a-97e338540f07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:18.346115 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b2b1b-71e5-4d40-bf9a-97e338540f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:18.346389 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b2b1b-71e5-4d40-bf9a-97e338540f07 admin admin] PATH_INFO: `/v3/users/86a6598d0d7c41faae54d48ea8a850d7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:18.347613 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b2b1b-71e5-4d40-bf9a-97e338540f07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86a6598d0d7c41faae54d48ea8a850d7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:18.349523 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b2b1b-71e5-4d40-bf9a-97e338540f07 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:18.401469 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dc6b2b1b-71e5-4d40-bf9a-97e338540f07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86a6598d0d7c41faae54d48ea8a850d7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:18.402128 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dc6b2b1b-71e5-4d40-bf9a-97e338540f07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86a6598d0d7c41faae54d48ea8a850d7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:18.402740 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dc6b2b1b-71e5-4d40-bf9a-97e338540f07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86a6598d0d7c41faae54d48ea8a850d7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:18.410079 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dc6b2b1b-71e5-4d40-bf9a-97e338540f07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86a6598d0d7c41faae54d48ea8a850d7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:18.412001 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:29:18.425369 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dc6b2b1b-71e5-4d40-bf9a-97e338540f07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86a6598d0d7c41faae54d48ea8a850d7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:18.440405 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1344/2695] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:29:18 2022] DELETE /identity/v3/users/86a6598d0d7c41faae54d48ea8a850d7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:29:18.626738 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7aa00-15c4-4385-b5fc-10e1bfcb6357 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:18.643173 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7aa00-15c4-4385-b5fc-10e1bfcb6357 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:18.732385 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5749f4-51a0-4020-9468-53aaa855b743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:18.734186 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5749f4-51a0-4020-9468-53aaa855b743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:18.734610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5749f4-51a0-4020-9468-53aaa855b743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:18.734984 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5749f4-51a0-4020-9468-53aaa855b743 admin admin] PATH_INFO: `/v3/projects/b5977f5c113f44eebed133afa6238c2e` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:18.738301 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5749f4-51a0-4020-9468-53aaa855b743 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5977f5c113f44eebed133afa6238c2e)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:18.740326 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5749f4-51a0-4020-9468-53aaa855b743 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:18.801469 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ad5749f4-51a0-4020-9468-53aaa855b743 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5977f5c113f44eebed133afa6238c2e', 'request_id': 'req-ad5749f4-51a0-4020-9468-53aaa855b743'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:18.813474 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ad5749f4-51a0-4020-9468-53aaa855b743 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5977f5c113f44eebed133afa6238c2e', 'request_id': 'req-ad5749f4-51a0-4020-9468-53aaa855b743'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:18.822055 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ad5749f4-51a0-4020-9468-53aaa855b743 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5977f5c113f44eebed133afa6238c2e', 'request_id': 'req-ad5749f4-51a0-4020-9468-53aaa855b743'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:18.838644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ad5749f4-51a0-4020-9468-53aaa855b743 admin admin] The token cache is being invalidate because project b5977f5c113f44eebed133afa6238c2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:29:18.838941 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ad5749f4-51a0-4020-9468-53aaa855b743 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:18.839623 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ad5749f4-51a0-4020-9468-53aaa855b743 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:18.841887 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1352/2696] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:29:18 2022] DELETE /identity/v3/projects/b5977f5c113f44eebed133afa6238c2e => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:29:18.845869 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b2b1b-71e5-4d40-bf9a-97e338540f07 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:18.847558 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dc6b2b1b-71e5-4d40-bf9a-97e338540f07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:18.866094 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b2b1b-71e5-4d40-bf9a-97e338540f07 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:18.941306 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8d072-f77d-4565-a850-f37e42a0b248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:18.942771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8d072-f77d-4565-a850-f37e42a0b248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:18.942771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8d072-f77d-4565-a850-f37e42a0b248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:18.942771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8d072-f77d-4565-a850-f37e42a0b248 admin admin] PATH_INFO: `/v3/users/e304d2e6991446ca833973980d124abc` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:18.947731 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd8d072-f77d-4565-a850-f37e42a0b248 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e304d2e6991446ca833973980d124abc)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:18.949669 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd8d072-f77d-4565-a850-f37e42a0b248 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:19.014075 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bbd8d072-f77d-4565-a850-f37e42a0b248 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e304d2e6991446ca833973980d124abc'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:19.015813 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bbd8d072-f77d-4565-a850-f37e42a0b248 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e304d2e6991446ca833973980d124abc'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:19.016666 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bbd8d072-f77d-4565-a850-f37e42a0b248 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e304d2e6991446ca833973980d124abc'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:19.026180 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bbd8d072-f77d-4565-a850-f37e42a0b248 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e304d2e6991446ca833973980d124abc'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:19.042522 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bbd8d072-f77d-4565-a850-f37e42a0b248 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e304d2e6991446ca833973980d124abc'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:19.067259 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1345/2697] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:29:18 2022] DELETE /identity/v3/users/e304d2e6991446ca833973980d124abc => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:29:19.418006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5749f4-51a0-4020-9468-53aaa855b743 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:19.419579 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ad5749f4-51a0-4020-9468-53aaa855b743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:19.444384 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5749f4-51a0-4020-9468-53aaa855b743 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:19.509570 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:29:19.525696 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d2df7-6634-4044-9443-ce174eb9149c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:19.526877 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d2df7-6634-4044-9443-ce174eb9149c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:19.527183 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d2df7-6634-4044-9443-ce174eb9149c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:19.527481 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d2df7-6634-4044-9443-ce174eb9149c admin admin] PATH_INFO: `/v3/projects/02c7033c26d04bdb8e8f091aefb1534b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:19.528681 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83d2df7-6634-4044-9443-ce174eb9149c admin admin] RBAC: Authorizing `identity:delete_project(project_id=02c7033c26d04bdb8e8f091aefb1534b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:19.531749 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83d2df7-6634-4044-9443-ce174eb9149c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:19.598671 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a83d2df7-6634-4044-9443-ce174eb9149c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02c7033c26d04bdb8e8f091aefb1534b', 'request_id': 'req-a83d2df7-6634-4044-9443-ce174eb9149c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:19.606827 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a83d2df7-6634-4044-9443-ce174eb9149c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02c7033c26d04bdb8e8f091aefb1534b', 'request_id': 'req-a83d2df7-6634-4044-9443-ce174eb9149c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:19.614189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a83d2df7-6634-4044-9443-ce174eb9149c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02c7033c26d04bdb8e8f091aefb1534b', 'request_id': 'req-a83d2df7-6634-4044-9443-ce174eb9149c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:19.625915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a83d2df7-6634-4044-9443-ce174eb9149c admin admin] The token cache is being invalidate because project 02c7033c26d04bdb8e8f091aefb1534b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:29:19.626204 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a83d2df7-6634-4044-9443-ce174eb9149c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:19.626877 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a83d2df7-6634-4044-9443-ce174eb9149c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:19.628457 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1353/2698] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:29:19 2022] DELETE /identity/v3/projects/02c7033c26d04bdb8e8f091aefb1534b => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:29:21.533695 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e29ff3-6b36-41ea-86ac-2210fd6a55e7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:21.534290 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e29ff3-6b36-41ea-86ac-2210fd6a55e7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:21.534612 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e29ff3-6b36-41ea-86ac-2210fd6a55e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:21.558217 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-78e29ff3-6b36-41ea-86ac-2210fd6a55e7 None None] MFA Rules not processed for user `d00b8bc311234e6da47b365ac8a7183a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:29:21.558217 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-78e29ff3-6b36-41ea-86ac-2210fd6a55e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:21.614346 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:29:21.625361 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1346/2699] 10.222.0.81 () {58 vars in 1009 bytes} [Tue Jun 7 14:29:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:29:21.674036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d2df7-6634-4044-9443-ce174eb9149c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:21.675496 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a83d2df7-6634-4044-9443-ce174eb9149c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:21.749293 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d2df7-6634-4044-9443-ce174eb9149c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:21.754700 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3696e71-07f0-45b5-823e-7fe0e227a071 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da7c0f99060a44eaab743ba71f159e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:21.754976 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3696e71-07f0-45b5-823e-7fe0e227a071 service swift] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:21.755286 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3696e71-07f0-45b5-823e-7fe0e227a071 service swift] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:21.755565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3696e71-07f0-45b5-823e-7fe0e227a071 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:21.764379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3696e71-07f0-45b5-823e-7fe0e227a071 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:21.766154 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3696e71-07f0-45b5-823e-7fe0e227a071 service swift] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:21.774641 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1354/2700] 10.222.0.81 () {60 vars in 1384 bytes} [Tue Jun 7 14:29:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:29:24.366423 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e29ff3-6b36-41ea-86ac-2210fd6a55e7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:24.368065 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-78e29ff3-6b36-41ea-86ac-2210fd6a55e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:24.472332 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e29ff3-6b36-41ea-86ac-2210fd6a55e7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:24.477526 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412a191-b010-4c98-8d52-2787d7a05e6e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab73cf3febbf4c768cd0116306f16c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:24.478291 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412a191-b010-4c98-8d52-2787d7a05e6e service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:24.478644 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412a191-b010-4c98-8d52-2787d7a05e6e service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:24.478988 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412a191-b010-4c98-8d52-2787d7a05e6e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:24.480956 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6412a191-b010-4c98-8d52-2787d7a05e6e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:24.494235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412a191-b010-4c98-8d52-2787d7a05e6e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:24.496007 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412a191-b010-4c98-8d52-2787d7a05e6e service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:24.515356 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1347/2701] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:29:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:29:31.771957 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=61,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:29:31.851730 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e573e4-597c-4411-9d7a-eafc49b80a1a None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:31.852019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e573e4-597c-4411-9d7a-eafc49b80a1a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:31.852303 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e573e4-597c-4411-9d7a-eafc49b80a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:31.874274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-b7e573e4-597c-4411-9d7a-eafc49b80a1a None None] MFA Rules not processed for user `d0b828a3dca14f2283a16894364d6eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:29:31.875615 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b7e573e4-597c-4411-9d7a-eafc49b80a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:31.950998 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1355/2702] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:29:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:29:32.391555 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb89c93-5a31-4eda-a449-59d5a75722d0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:32.392064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb89c93-5a31-4eda-a449-59d5a75722d0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:32.392164 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb89c93-5a31-4eda-a449-59d5a75722d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:32.411452 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9fb89c93-5a31-4eda-a449-59d5a75722d0 None None] MFA Rules not processed for user `d00b8bc311234e6da47b365ac8a7183a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:29:32.412893 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9fb89c93-5a31-4eda-a449-59d5a75722d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:32.421103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1348/2703] 10.222.0.81 () {58 vars in 1009 bytes} [Tue Jun 7 14:29:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:29:32.433632 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e573e4-597c-4411-9d7a-eafc49b80a1a None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:32.445983 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e573e4-597c-4411-9d7a-eafc49b80a1a None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:32.451102 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629207c-07fc-4957-8950-c11497e83a46 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da7c0f99060a44eaab743ba71f159e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:32.452537 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629207c-07fc-4957-8950-c11497e83a46 service swift] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:32.452819 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629207c-07fc-4957-8950-c11497e83a46 service swift] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:32.453113 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629207c-07fc-4957-8950-c11497e83a46 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:32.461629 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a629207c-07fc-4957-8950-c11497e83a46 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:32.463366 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a629207c-07fc-4957-8950-c11497e83a46 service swift] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:32.472074 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1356/2704] 10.222.0.81 () {60 vars in 1384 bytes} [Tue Jun 7 14:29:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:29:33.048446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f4500-c213-48a4-a69d-d5029495e236 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:33.048938 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f4500-c213-48a4-a69d-d5029495e236 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:33.049216 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f4500-c213-48a4-a69d-d5029495e236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:33.067573 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-cf7f4500-c213-48a4-a69d-d5029495e236 None None] MFA Rules not processed for user `d00b8bc311234e6da47b365ac8a7183a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:29:33.068908 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cf7f4500-c213-48a4-a69d-d5029495e236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:33.077640 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1349/2705] 10.222.0.81 () {58 vars in 967 bytes} [Tue Jun 7 14:29:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:29:33.092502 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629207c-07fc-4957-8950-c11497e83a46 service swift] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:33.104969 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629207c-07fc-4957-8950-c11497e83a46 service swift] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:33.111408 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18679ff-01b5-4a6b-a752-a5b1b5bb88cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da7c0f99060a44eaab743ba71f159e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:33.112371 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18679ff-01b5-4a6b-a752-a5b1b5bb88cc service swift] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:33.112898 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18679ff-01b5-4a6b-a752-a5b1b5bb88cc service swift] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:33.113268 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18679ff-01b5-4a6b-a752-a5b1b5bb88cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:33.122744 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18679ff-01b5-4a6b-a752-a5b1b5bb88cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:33.125187 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18679ff-01b5-4a6b-a752-a5b1b5bb88cc service swift] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:33.135759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1357/2706] 10.222.0.81 () {60 vars in 1384 bytes} [Tue Jun 7 14:29:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:29:33.389883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc60e9-2af6-4c88-a3d3-156f89585c1b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:33.390188 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc60e9-2af6-4c88-a3d3-156f89585c1b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:33.390454 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc60e9-2af6-4c88-a3d3-156f89585c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:33.408853 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d9fc60e9-2af6-4c88-a3d3-156f89585c1b None None] MFA Rules not processed for user `d00b8bc311234e6da47b365ac8a7183a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:29:33.410068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d9fc60e9-2af6-4c88-a3d3-156f89585c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:33.419470 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1350/2707] 10.222.0.81 () {58 vars in 1009 bytes} [Tue Jun 7 14:29:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:29:33.430492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18679ff-01b5-4a6b-a752-a5b1b5bb88cc service swift] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:33.445105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18679ff-01b5-4a6b-a752-a5b1b5bb88cc service swift] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:33.450483 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c15844-e15a-4e7c-a58b-af0f23ea8a0a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da7c0f99060a44eaab743ba71f159e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:33.451806 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c15844-e15a-4e7c-a58b-af0f23ea8a0a service swift] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:33.452362 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c15844-e15a-4e7c-a58b-af0f23ea8a0a service swift] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:33.452705 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c15844-e15a-4e7c-a58b-af0f23ea8a0a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:33.461678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c15844-e15a-4e7c-a58b-af0f23ea8a0a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:33.463352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c15844-e15a-4e7c-a58b-af0f23ea8a0a service swift] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:33.475388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1358/2708] 10.222.0.81 () {60 vars in 1384 bytes} [Tue Jun 7 14:29:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:29:33.807747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ee8f8-f589-4bc4-b437-542071801034 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:33.808034 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ee8f8-f589-4bc4-b437-542071801034 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:33.808308 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ee8f8-f589-4bc4-b437-542071801034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:33.828403 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-030ee8f8-f589-4bc4-b437-542071801034 None None] MFA Rules not processed for user `d00b8bc311234e6da47b365ac8a7183a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:29:33.829587 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-030ee8f8-f589-4bc4-b437-542071801034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:33.836779 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1351/2709] 10.222.0.81 () {58 vars in 967 bytes} [Tue Jun 7 14:29:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:29:33.849209 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c15844-e15a-4e7c-a58b-af0f23ea8a0a service swift] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:33.859917 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c15844-e15a-4e7c-a58b-af0f23ea8a0a service swift] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:33.864906 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a07598e-f54b-41d4-95b9-f488be00843f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da7c0f99060a44eaab743ba71f159e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:33.865541 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07598e-f54b-41d4-95b9-f488be00843f service swift] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:33.865826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07598e-f54b-41d4-95b9-f488be00843f service swift] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:33.866105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07598e-f54b-41d4-95b9-f488be00843f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:33.874867 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a07598e-f54b-41d4-95b9-f488be00843f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:33.876517 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a07598e-f54b-41d4-95b9-f488be00843f service swift] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:33.884763 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1359/2710] 10.222.0.81 () {60 vars in 1384 bytes} [Tue Jun 7 14:29:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 07 14:29:35.973287 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ee8f8-f589-4bc4-b437-542071801034 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:35.974927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-030ee8f8-f589-4bc4-b437-542071801034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:36.083984 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ee8f8-f589-4bc4-b437-542071801034 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:36.088756 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b1dae6-d9a3-448a-8d8d-03e348e707ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:36.089899 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1dae6-d9a3-448a-8d8d-03e348e707ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:36.090339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1dae6-d9a3-448a-8d8d-03e348e707ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:36.090800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1dae6-d9a3-448a-8d8d-03e348e707ed admin admin] PATH_INFO: `/v3/users/4185a7263943447a92a15571f192b0f8` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:36.092635 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b1dae6-d9a3-448a-8d8d-03e348e707ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=4185a7263943447a92a15571f192b0f8)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:36.094557 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b1dae6-d9a3-448a-8d8d-03e348e707ed admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:36.151957 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c1b1dae6-d9a3-448a-8d8d-03e348e707ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4185a7263943447a92a15571f192b0f8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:36.152906 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c1b1dae6-d9a3-448a-8d8d-03e348e707ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4185a7263943447a92a15571f192b0f8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:36.153535 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c1b1dae6-d9a3-448a-8d8d-03e348e707ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4185a7263943447a92a15571f192b0f8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:36.164427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c1b1dae6-d9a3-448a-8d8d-03e348e707ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4185a7263943447a92a15571f192b0f8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:36.176433 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:29:36.183357 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c1b1dae6-d9a3-448a-8d8d-03e348e707ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4185a7263943447a92a15571f192b0f8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:36.198893 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1352/2711] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:29:35 2022] DELETE /identity/v3/users/4185a7263943447a92a15571f192b0f8 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:29:36.203910 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9193f-ba09-4c7c-a3e8-293d49a43d6e None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:36.208287 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9193f-ba09-4c7c-a3e8-293d49a43d6e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:36.208598 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9193f-ba09-4c7c-a3e8-293d49a43d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:36.230262 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-3ea9193f-ba09-4c7c-a3e8-293d49a43d6e None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:29:36.231249 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3ea9193f-ba09-4c7c-a3e8-293d49a43d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:36.257355 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8529e6-3d72-47e4-a140-d3dd63f14de1 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:36.258039 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8529e6-3d72-47e4-a140-d3dd63f14de1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:36.258513 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8529e6-3d72-47e4-a140-d3dd63f14de1 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:36.261026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1353/2712] 10.222.0.81 () {54 vars in 921 bytes} [Tue Jun 7 14:29:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:29:36.269084 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafa15d-d1da-4b79-a69f-7b0f9a4abb57 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:36.269617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafa15d-d1da-4b79-a69f-7b0f9a4abb57 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:36.270054 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafa15d-d1da-4b79-a69f-7b0f9a4abb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:36.302116 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7fafa15d-d1da-4b79-a69f-7b0f9a4abb57 None None] MFA Rules not processed for user `2ae726fbc9cf40e68c7587552a118a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:29:36.303923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7fafa15d-d1da-4b79-a69f-7b0f9a4abb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:36.326350 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1360/2713] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:29:36.338362 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9193f-ba09-4c7c-a3e8-293d49a43d6e None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:36.340146 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3ea9193f-ba09-4c7c-a3e8-293d49a43d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:36.416570 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1354/2714] 10.222.0.81 () {58 vars in 1018 bytes} [Tue Jun 7 14:29:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:29:36.430690 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fafa15d-d1da-4b79-a69f-7b0f9a4abb57 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:36.432751 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7fafa15d-d1da-4b79-a69f-7b0f9a4abb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:36.433394 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9193f-ba09-4c7c-a3e8-293d49a43d6e None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:36.438531 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224912fb-e564-49c4-9a9b-dc61b4169712 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:36.439638 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224912fb-e564-49c4-9a9b-dc61b4169712 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:36.440021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224912fb-e564-49c4-9a9b-dc61b4169712 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:36.440460 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224912fb-e564-49c4-9a9b-dc61b4169712 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:36.451546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:29:36.455827 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224912fb-e564-49c4-9a9b-dc61b4169712 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:36.458841 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224912fb-e564-49c4-9a9b-dc61b4169712 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:36.499778 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1361/2715] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:29:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:29:36.528327 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fafa15d-d1da-4b79-a69f-7b0f9a4abb57 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:36.532532 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888e7fb-4333-49fa-a9a0-0688623a68a5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6180393da0f495db964f0b2588d8b90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:36.533206 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888e7fb-4333-49fa-a9a0-0688623a68a5 service glance] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:36.533491 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888e7fb-4333-49fa-a9a0-0688623a68a5 service glance] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:36.533769 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888e7fb-4333-49fa-a9a0-0688623a68a5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:36.548340 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888e7fb-4333-49fa-a9a0-0688623a68a5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:36.550708 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888e7fb-4333-49fa-a9a0-0688623a68a5 service glance] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:36.565715 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1355/2716] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:29:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:29:36.749503 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224912fb-e564-49c4-9a9b-dc61b4169712 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:36.751263 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-224912fb-e564-49c4-9a9b-dc61b4169712 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:36.767935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224912fb-e564-49c4-9a9b-dc61b4169712 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:36.773416 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea541d-9bf1-47d6-8255-22dde2560cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:36.774367 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea541d-9bf1-47d6-8255-22dde2560cc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:36.774654 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea541d-9bf1-47d6-8255-22dde2560cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:36.774931 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea541d-9bf1-47d6-8255-22dde2560cc8 admin admin] PATH_INFO: `/v3/projects/6e8cfd2bd8134c328696944b1913bc41` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:36.777444 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea541d-9bf1-47d6-8255-22dde2560cc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e8cfd2bd8134c328696944b1913bc41)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:36.780158 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea541d-9bf1-47d6-8255-22dde2560cc8 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:36.840909 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c3ea541d-9bf1-47d6-8255-22dde2560cc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e8cfd2bd8134c328696944b1913bc41', 'request_id': 'req-c3ea541d-9bf1-47d6-8255-22dde2560cc8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:36.868699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c3ea541d-9bf1-47d6-8255-22dde2560cc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e8cfd2bd8134c328696944b1913bc41', 'request_id': 'req-c3ea541d-9bf1-47d6-8255-22dde2560cc8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:36.880144 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c3ea541d-9bf1-47d6-8255-22dde2560cc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e8cfd2bd8134c328696944b1913bc41', 'request_id': 'req-c3ea541d-9bf1-47d6-8255-22dde2560cc8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:36.892879 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c3ea541d-9bf1-47d6-8255-22dde2560cc8 admin admin] The token cache is being invalidate because project 6e8cfd2bd8134c328696944b1913bc41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:29:36.893202 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c3ea541d-9bf1-47d6-8255-22dde2560cc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:36.893881 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c3ea541d-9bf1-47d6-8255-22dde2560cc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:29:36.895684 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1362/2717] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:29:36 2022] DELETE /identity/v3/projects/6e8cfd2bd8134c328696944b1913bc41 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:29:39.118025 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb00ec8-c582-4bff-85ad-3f937021d80e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:39.118498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb00ec8-c582-4bff-85ad-3f937021d80e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:39.118593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb00ec8-c582-4bff-85ad-3f937021d80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:39.136297 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-cbb00ec8-c582-4bff-85ad-3f937021d80e None None] MFA Rules not processed for user `d00b8bc311234e6da47b365ac8a7183a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:29:39.137561 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cbb00ec8-c582-4bff-85ad-3f937021d80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:39.193002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1356/2718] 10.222.0.81 () {58 vars in 1009 bytes} [Tue Jun 7 14:29:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:29:39.204610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea541d-9bf1-47d6-8255-22dde2560cc8 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:29:39.206021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c3ea541d-9bf1-47d6-8255-22dde2560cc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:29:39.281814 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea541d-9bf1-47d6-8255-22dde2560cc8 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:29:39.287228 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f5dc8-1f60-494f-a1af-e27ba40e9d9a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da7c0f99060a44eaab743ba71f159e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:29:39.287914 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f5dc8-1f60-494f-a1af-e27ba40e9d9a service swift] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:29:39.288196 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f5dc8-1f60-494f-a1af-e27ba40e9d9a service swift] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:29:39.288468 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f5dc8-1f60-494f-a1af-e27ba40e9d9a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:29:39.298063 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f5dc8-1f60-494f-a1af-e27ba40e9d9a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:29:39.299870 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f5dc8-1f60-494f-a1af-e27ba40e9d9a service swift] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:29:39.308267 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1363/2719] 10.222.0.81 () {60 vars in 1384 bytes} [Tue Jun 7 14:29:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:29:49.186445 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=87,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:29:49.304866 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=61,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:30:00.349436 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f39a91-2b99-427e-9106-30773c197040 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:30:00.349947 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f39a91-2b99-427e-9106-30773c197040 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:30:00.350150 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f39a91-2b99-427e-9106-30773c197040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:30:00.368451 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-91f39a91-2b99-427e-9106-30773c197040 None None] MFA Rules not processed for user `d00b8bc311234e6da47b365ac8a7183a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:30:00.369703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-91f39a91-2b99-427e-9106-30773c197040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:30:00.377007 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1357/2720] 10.222.0.81 () {58 vars in 967 bytes} [Tue Jun 7 14:30:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:30:00.390729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f5dc8-1f60-494f-a1af-e27ba40e9d9a service swift] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:30:00.401610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f5dc8-1f60-494f-a1af-e27ba40e9d9a service swift] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:30:00.407501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516b54e-3f40-4a3c-b3c5-8769e8ac1f47 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da7c0f99060a44eaab743ba71f159e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:30:00.408137 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516b54e-3f40-4a3c-b3c5-8769e8ac1f47 service swift] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:30:00.408419 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516b54e-3f40-4a3c-b3c5-8769e8ac1f47 service swift] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:30:00.408693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516b54e-3f40-4a3c-b3c5-8769e8ac1f47 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:30:00.416773 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516b54e-3f40-4a3c-b3c5-8769e8ac1f47 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:30:00.418323 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516b54e-3f40-4a3c-b3c5-8769e8ac1f47 service swift] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:30:00.426309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1364/2721] 10.222.0.81 () {60 vars in 1384 bytes} [Tue Jun 7 14:30:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:30:10.362851 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:30:10.423203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=6 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:30:41.888836 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce110f0a-619f-49be-9148-cfe068065319 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:30:41.889365 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce110f0a-619f-49be-9148-cfe068065319 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:30:41.889466 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce110f0a-619f-49be-9148-cfe068065319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:30:41.908368 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ce110f0a-619f-49be-9148-cfe068065319 None None] MFA Rules not processed for user `fefdd9dfa11641a598fcc29b2ad32afc`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:30:41.909575 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ce110f0a-619f-49be-9148-cfe068065319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:30:41.966269 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1358/2722] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:30:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:30:51.959462 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=25 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:31:31.906338 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516b54e-3f40-4a3c-b3c5-8769e8ac1f47 service swift] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:31:31.907795 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8516b54e-3f40-4a3c-b3c5-8769e8ac1f47 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:31.963905 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516b54e-3f40-4a3c-b3c5-8769e8ac1f47 service swift] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:31:31.969647 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dfab9d-ce89-4e76-9286-2a176f960416 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6180393da0f495db964f0b2588d8b90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:31:31.970296 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfab9d-ce89-4e76-9286-2a176f960416 service glance] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:31.970739 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfab9d-ce89-4e76-9286-2a176f960416 service glance] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:31.971025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfab9d-ce89-4e76-9286-2a176f960416 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:31.974175 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-40dfab9d-ce89-4e76-9286-2a176f960416 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:32.056365 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dfab9d-ce89-4e76-9286-2a176f960416 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:31:32.057727 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dfab9d-ce89-4e76-9286-2a176f960416 service glance] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:31:32.068943 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1365/2723] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:31:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:31:32.311973 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce110f0a-619f-49be-9148-cfe068065319 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:31:32.313430 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ce110f0a-619f-49be-9148-cfe068065319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:32.370201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce110f0a-619f-49be-9148-cfe068065319 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:31:32.373777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85566e17-1dfa-427e-887f-93c26144481a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:31:32.374420 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85566e17-1dfa-427e-887f-93c26144481a service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:32.374702 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85566e17-1dfa-427e-887f-93c26144481a service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:32.374977 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85566e17-1dfa-427e-887f-93c26144481a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:32.376757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-85566e17-1dfa-427e-887f-93c26144481a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:32.390478 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85566e17-1dfa-427e-887f-93c26144481a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:31:32.391924 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85566e17-1dfa-427e-887f-93c26144481a service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:31:32.403369 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1359/2724] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:31:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:31:37.489135 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dfab9d-ce89-4e76-9286-2a176f960416 service glance] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:31:37.490606 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-40dfab9d-ce89-4e76-9286-2a176f960416 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:37.552835 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dfab9d-ce89-4e76-9286-2a176f960416 service glance] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:31:37.556384 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe5e1f-087b-46f7-96a3-90daa447a780 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:31:37.557023 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe5e1f-087b-46f7-96a3-90daa447a780 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:37.557304 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe5e1f-087b-46f7-96a3-90daa447a780 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:37.557575 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe5e1f-087b-46f7-96a3-90daa447a780 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:37.559334 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-54fe5e1f-087b-46f7-96a3-90daa447a780 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:37.615546 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:31:37.617737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe5e1f-087b-46f7-96a3-90daa447a780 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:31:37.619500 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe5e1f-087b-46f7-96a3-90daa447a780 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:31:37.628542 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1366/2725] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:31:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:31:42.398503 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=26 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:31:43.247162 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817bbab-e464-48c8-8dba-176dc86c43f5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:43.247498 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817bbab-e464-48c8-8dba-176dc86c43f5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:43.247777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817bbab-e464-48c8-8dba-176dc86c43f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:43.266407 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c817bbab-e464-48c8-8dba-176dc86c43f5 None None] MFA Rules not processed for user `d0b828a3dca14f2283a16894364d6eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:31:43.267654 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c817bbab-e464-48c8-8dba-176dc86c43f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:43.275578 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1360/2726] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:31:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:31:45.792234 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe5e1f-087b-46f7-96a3-90daa447a780 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:31:45.793847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-54fe5e1f-087b-46f7-96a3-90daa447a780 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:45.807241 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe5e1f-087b-46f7-96a3-90daa447a780 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:31:45.811145 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb7a50-de23-4408-a701-ec45c205ffea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:31:45.811851 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb7a50-de23-4408-a701-ec45c205ffea service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:45.812135 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb7a50-de23-4408-a701-ec45c205ffea service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:45.812432 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb7a50-de23-4408-a701-ec45c205ffea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:45.814210 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2fdb7a50-de23-4408-a701-ec45c205ffea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:45.825545 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb7a50-de23-4408-a701-ec45c205ffea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:31:45.826994 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb7a50-de23-4408-a701-ec45c205ffea service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:31:45.837253 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1367/2727] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:31:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:31:46.483782 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817bbab-e464-48c8-8dba-176dc86c43f5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:31:46.486368 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c817bbab-e464-48c8-8dba-176dc86c43f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:46.535113 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817bbab-e464-48c8-8dba-176dc86c43f5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:31:46.544307 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f07cfb-74d1-4403-b0c3-cfd201b8c15c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:31:46.545365 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f07cfb-74d1-4403-b0c3-cfd201b8c15c service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:46.545858 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f07cfb-74d1-4403-b0c3-cfd201b8c15c service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:46.546369 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f07cfb-74d1-4403-b0c3-cfd201b8c15c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:46.548806 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e1f07cfb-74d1-4403-b0c3-cfd201b8c15c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:46.566302 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f07cfb-74d1-4403-b0c3-cfd201b8c15c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:31:46.569254 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f07cfb-74d1-4403-b0c3-cfd201b8c15c service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:31:46.584753 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1361/2728] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:31:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:31:50.234770 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8f4cd-9b13-4a59-838f-9babb542eb8e None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:50.235342 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8f4cd-9b13-4a59-838f-9babb542eb8e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:50.235455 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8f4cd-9b13-4a59-838f-9babb542eb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:50.255254 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-5fe8f4cd-9b13-4a59-838f-9babb542eb8e None None] MFA Rules not processed for user `d00b8bc311234e6da47b365ac8a7183a`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:31:50.256560 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5fe8f4cd-9b13-4a59-838f-9babb542eb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:50.263970 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1368/2729] 10.222.0.81 () {58 vars in 1009 bytes} [Tue Jun 7 14:31:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:31:50.319469 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f07cfb-74d1-4403-b0c3-cfd201b8c15c service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:31:50.331448 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f07cfb-74d1-4403-b0c3-cfd201b8c15c service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:31:50.337039 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6758c9-3466-4769-bdb7-0ed264540c8a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da7c0f99060a44eaab743ba71f159e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:31:50.337783 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6758c9-3466-4769-bdb7-0ed264540c8a service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:50.338139 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6758c9-3466-4769-bdb7-0ed264540c8a service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:50.338490 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6758c9-3466-4769-bdb7-0ed264540c8a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:50.347781 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6758c9-3466-4769-bdb7-0ed264540c8a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:31:50.349385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6758c9-3466-4769-bdb7-0ed264540c8a service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:31:50.369489 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1362/2730] 10.222.0.81 () {60 vars in 1384 bytes} [Tue Jun 7 14:31:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:32:00.249231 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=11 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:32:00.354779 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=15 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:32:04.443304 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116ba26-4cf3-43b4-8bf9-59c41e765ac3 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:04.443880 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116ba26-4cf3-43b4-8bf9-59c41e765ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:04.443880 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116ba26-4cf3-43b4-8bf9-59c41e765ac3 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:04.445227 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1369/2731] 10.222.0.81 () {54 vars in 921 bytes} [Tue Jun 7 14:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:32:04.451801 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdbde1-d7c5-43a1-89db-aaecfdc9fe63 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:04.452286 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdbde1-d7c5-43a1-89db-aaecfdc9fe63 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:04.452406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdbde1-d7c5-43a1-89db-aaecfdc9fe63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:04.470706 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-76fdbde1-d7c5-43a1-89db-aaecfdc9fe63 None None] MFA Rules not processed for user `2ae726fbc9cf40e68c7587552a118a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:04.471935 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-76fdbde1-d7c5-43a1-89db-aaecfdc9fe63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:04.481569 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1363/2732] 10.222.0.81 () {58 vars in 1018 bytes} [Tue Jun 7 14:32:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:32:04.494112 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8116ba26-4cf3-43b4-8bf9-59c41e765ac3 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:04.505373 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8116ba26-4cf3-43b4-8bf9-59c41e765ac3 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:04.509036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9280759-3c32-4d46-b4f1-ff3c66edc7b1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6180393da0f495db964f0b2588d8b90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:04.509677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9280759-3c32-4d46-b4f1-ff3c66edc7b1 service glance] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:04.509960 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9280759-3c32-4d46-b4f1-ff3c66edc7b1 service glance] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:04.510236 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9280759-3c32-4d46-b4f1-ff3c66edc7b1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:04.521407 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9280759-3c32-4d46-b4f1-ff3c66edc7b1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:04.522862 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9280759-3c32-4d46-b4f1-ff3c66edc7b1 service glance] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:04.534013 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1370/2733] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:32:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:32:04.603155 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86959d5c-c33a-43bb-8d79-6cb2403c80a8 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:04.603468 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86959d5c-c33a-43bb-8d79-6cb2403c80a8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:04.603744 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86959d5c-c33a-43bb-8d79-6cb2403c80a8 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:04.605111 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1364/2734] 10.222.0.81 () {54 vars in 921 bytes} [Tue Jun 7 14:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:32:04.609734 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d089f-9690-4af9-a7a8-ae5743c327dc None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:04.610129 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d089f-9690-4af9-a7a8-ae5743c327dc None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:04.610409 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d089f-9690-4af9-a7a8-ae5743c327dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:04.647105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-d96d089f-9690-4af9-a7a8-ae5743c327dc None None] MFA Rules not processed for user `a6a08eaecbcd41228571739d8387882a`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:04.648327 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d96d089f-9690-4af9-a7a8-ae5743c327dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:04.654923 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1371/2735] 10.222.0.81 () {58 vars in 1018 bytes} [Tue Jun 7 14:32:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:32:04.666997 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86959d5c-c33a-43bb-8d79-6cb2403c80a8 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:04.678572 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86959d5c-c33a-43bb-8d79-6cb2403c80a8 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:04.682164 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ed415-f6bc-4cba-988d-adb23aa5e316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:04.682802 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ed415-f6bc-4cba-988d-adb23aa5e316 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:04.683083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ed415-f6bc-4cba-988d-adb23aa5e316 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:04.683418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ed415-f6bc-4cba-988d-adb23aa5e316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:04.690470 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ed415-f6bc-4cba-988d-adb23aa5e316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:04.691931 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ed415-f6bc-4cba-988d-adb23aa5e316 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:04.699524 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1365/2736] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:32:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:32:04.786415 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba05d66-e7ab-4fb0-9110-e9f34b838f82 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:04.786703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba05d66-e7ab-4fb0-9110-e9f34b838f82 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:04.786980 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba05d66-e7ab-4fb0-9110-e9f34b838f82 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:04.788395 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1372/2737] 10.222.0.81 () {54 vars in 921 bytes} [Tue Jun 7 14:32:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:32:04.793643 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364d68d-7680-4d4b-bba6-64694feb720d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:04.793933 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364d68d-7680-4d4b-bba6-64694feb720d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:04.794211 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364d68d-7680-4d4b-bba6-64694feb720d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:04.812181 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d364d68d-7680-4d4b-bba6-64694feb720d None None] MFA Rules not processed for user `2ae726fbc9cf40e68c7587552a118a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:04.813373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d364d68d-7680-4d4b-bba6-64694feb720d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:04.822593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1366/2738] 10.222.0.81 () {58 vars in 1018 bytes} [Tue Jun 7 14:32:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:32:04.833239 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba05d66-e7ab-4fb0-9110-e9f34b838f82 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:04.844213 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba05d66-e7ab-4fb0-9110-e9f34b838f82 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:04.848046 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97982c-b1bd-4920-bbba-0b89c0e72cc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:04.848723 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97982c-b1bd-4920-bbba-0b89c0e72cc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:04.849008 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97982c-b1bd-4920-bbba-0b89c0e72cc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:04.849284 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97982c-b1bd-4920-bbba-0b89c0e72cc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:04.860514 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97982c-b1bd-4920-bbba-0b89c0e72cc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:04.862104 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97982c-b1bd-4920-bbba-0b89c0e72cc2 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:04.874065 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1373/2739] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:32:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:32:04.934789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364d68d-7680-4d4b-bba6-64694feb720d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:04.936447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d364d68d-7680-4d4b-bba6-64694feb720d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:05.004043 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364d68d-7680-4d4b-bba6-64694feb720d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:05.009936 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2984dfc-d4ef-41ce-973a-d9f9211b18c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab73cf3febbf4c768cd0116306f16c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:05.010579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2984dfc-d4ef-41ce-973a-d9f9211b18c1 service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:05.010856 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2984dfc-d4ef-41ce-973a-d9f9211b18c1 service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:05.011128 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2984dfc-d4ef-41ce-973a-d9f9211b18c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:05.012921 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b2984dfc-d4ef-41ce-973a-d9f9211b18c1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:05.024295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2984dfc-d4ef-41ce-973a-d9f9211b18c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:05.025854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2984dfc-d4ef-41ce-973a-d9f9211b18c1 service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:05.036046 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1367/2740] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:32:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:32:06.512404 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5662b0f-60b7-4898-aef2-889e8a50b97f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:06.512906 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5662b0f-60b7-4898-aef2-889e8a50b97f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:06.513009 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5662b0f-60b7-4898-aef2-889e8a50b97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:06.530287 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-d5662b0f-60b7-4898-aef2-889e8a50b97f None None] MFA Rules not processed for user `d00b8bc311234e6da47b365ac8a7183a`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:06.531502 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d5662b0f-60b7-4898-aef2-889e8a50b97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:06.538721 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1374/2741] 10.222.0.81 () {58 vars in 1009 bytes} [Tue Jun 7 14:32:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:32:06.550371 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2984dfc-d4ef-41ce-973a-d9f9211b18c1 service placement] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:06.561990 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2984dfc-d4ef-41ce-973a-d9f9211b18c1 service placement] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:06.566852 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf6e29-41f8-4878-ad66-7b903e597efa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da7c0f99060a44eaab743ba71f159e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:06.568200 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf6e29-41f8-4878-ad66-7b903e597efa service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:06.568482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf6e29-41f8-4878-ad66-7b903e597efa service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:06.568752 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf6e29-41f8-4878-ad66-7b903e597efa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:06.576810 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf6e29-41f8-4878-ad66-7b903e597efa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:06.578443 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf6e29-41f8-4878-ad66-7b903e597efa service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:06.586525 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1368/2742] 10.222.0.81 () {60 vars in 1384 bytes} [Tue Jun 7 14:32:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:32:16.525278 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=24 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:32:16.583278 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=50 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:32:19.609336 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5662b0f-60b7-4898-aef2-889e8a50b97f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:19.623676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5662b0f-60b7-4898-aef2-889e8a50b97f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:19.629109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d116d8-5e2a-4ab3-9db5-52dcb244c0fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:19.629760 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d116d8-5e2a-4ab3-9db5-52dcb244c0fd service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:19.630045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d116d8-5e2a-4ab3-9db5-52dcb244c0fd service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:19.630322 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d116d8-5e2a-4ab3-9db5-52dcb244c0fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:19.633290 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-04d116d8-5e2a-4ab3-9db5-52dcb244c0fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:19.645849 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d116d8-5e2a-4ab3-9db5-52dcb244c0fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:19.647594 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d116d8-5e2a-4ab3-9db5-52dcb244c0fd service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:19.658537 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1375/2743] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:32:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:32:21.054611 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0329fe-e479-43e2-89bb-fc4cafa7f871 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:21.055113 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0329fe-e479-43e2-89bb-fc4cafa7f871 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:21.055254 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0329fe-e479-43e2-89bb-fc4cafa7f871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:21.073925 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2e0329fe-e479-43e2-89bb-fc4cafa7f871 None None] MFA Rules not processed for user `d00b8bc311234e6da47b365ac8a7183a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:21.075195 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2e0329fe-e479-43e2-89bb-fc4cafa7f871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:21.082633 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1369/2744] 10.222.0.81 () {58 vars in 967 bytes} [Tue Jun 7 14:32:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:32:21.096062 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d116d8-5e2a-4ab3-9db5-52dcb244c0fd service nova] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:21.108467 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d116d8-5e2a-4ab3-9db5-52dcb244c0fd service nova] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:21.113474 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2986804f-4c2e-4cc7-84b0-53d245fecd4a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da7c0f99060a44eaab743ba71f159e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:21.114117 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986804f-4c2e-4cc7-84b0-53d245fecd4a service swift] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:21.114399 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986804f-4c2e-4cc7-84b0-53d245fecd4a service swift] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:21.114674 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986804f-4c2e-4cc7-84b0-53d245fecd4a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:21.124002 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2986804f-4c2e-4cc7-84b0-53d245fecd4a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:21.125575 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2986804f-4c2e-4cc7-84b0-53d245fecd4a service swift] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:21.134404 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1376/2745] 10.222.0.81 () {60 vars in 1384 bytes} [Tue Jun 7 14:32:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:32:31.067466 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:32:31.130404 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=12 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:32:59.680673 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0329fe-e479-43e2-89bb-fc4cafa7f871 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:59.682669 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2e0329fe-e479-43e2-89bb-fc4cafa7f871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:59.693851 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0329fe-e479-43e2-89bb-fc4cafa7f871 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:59.697664 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a20cc-36eb-4958-84f0-e55600b22daf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:59.698304 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a20cc-36eb-4958-84f0-e55600b22daf service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:59.698585 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a20cc-36eb-4958-84f0-e55600b22daf service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:59.698857 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a20cc-36eb-4958-84f0-e55600b22daf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:59.700565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-944a20cc-36eb-4958-84f0-e55600b22daf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:59.710844 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a20cc-36eb-4958-84f0-e55600b22daf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:59.712304 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a20cc-36eb-4958-84f0-e55600b22daf service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:59.721485 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1370/2746] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:32:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:33:02.121428 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2986804f-4c2e-4cc7-84b0-53d245fecd4a service swift] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:02.130755 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2986804f-4c2e-4cc7-84b0-53d245fecd4a service swift] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:02.134355 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c72f79-da7a-4b0b-8e9b-4c3aaf5ca040 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:02.134992 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c72f79-da7a-4b0b-8e9b-4c3aaf5ca040 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:02.135301 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c72f79-da7a-4b0b-8e9b-4c3aaf5ca040 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:02.135585 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c72f79-da7a-4b0b-8e9b-4c3aaf5ca040 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:02.137339 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-31c72f79-da7a-4b0b-8e9b-4c3aaf5ca040 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:02.200194 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c72f79-da7a-4b0b-8e9b-4c3aaf5ca040 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:02.201656 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c72f79-da7a-4b0b-8e9b-4c3aaf5ca040 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:02.211043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1377/2747] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:33:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:33:05.268170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a20cc-36eb-4958-84f0-e55600b22daf service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:05.279931 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a20cc-36eb-4958-84f0-e55600b22daf service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:05.285291 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d90990-7dec-46c7-b098-7ed24cdfac8f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab73cf3febbf4c768cd0116306f16c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:05.286093 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d90990-7dec-46c7-b098-7ed24cdfac8f service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:05.286442 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d90990-7dec-46c7-b098-7ed24cdfac8f service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:05.286834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d90990-7dec-46c7-b098-7ed24cdfac8f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:05.288842 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d6d90990-7dec-46c7-b098-7ed24cdfac8f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:05.303351 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d90990-7dec-46c7-b098-7ed24cdfac8f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:05.304928 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d90990-7dec-46c7-b098-7ed24cdfac8f service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:05.316410 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1371/2748] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:33:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:33:05.322647 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c72f79-da7a-4b0b-8e9b-4c3aaf5ca040 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:05.324308 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-31c72f79-da7a-4b0b-8e9b-4c3aaf5ca040 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:05.347797 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c72f79-da7a-4b0b-8e9b-4c3aaf5ca040 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:05.357728 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126079c1-bab9-492d-aab0-66f2c9ed4936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:05.358959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126079c1-bab9-492d-aab0-66f2c9ed4936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:05.359324 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126079c1-bab9-492d-aab0-66f2c9ed4936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:05.359772 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126079c1-bab9-492d-aab0-66f2c9ed4936 admin admin] PATH_INFO: `/v3/users/fefdd9dfa11641a598fcc29b2ad32afc` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:05.361066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126079c1-bab9-492d-aab0-66f2c9ed4936 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fefdd9dfa11641a598fcc29b2ad32afc)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:05.363519 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126079c1-bab9-492d-aab0-66f2c9ed4936 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:05.409615 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-126079c1-bab9-492d-aab0-66f2c9ed4936 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fefdd9dfa11641a598fcc29b2ad32afc'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:05.421792 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-126079c1-bab9-492d-aab0-66f2c9ed4936 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fefdd9dfa11641a598fcc29b2ad32afc'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:05.422471 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-126079c1-bab9-492d-aab0-66f2c9ed4936 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fefdd9dfa11641a598fcc29b2ad32afc'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:05.423060 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-126079c1-bab9-492d-aab0-66f2c9ed4936 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fefdd9dfa11641a598fcc29b2ad32afc'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:05.429696 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-126079c1-bab9-492d-aab0-66f2c9ed4936 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fefdd9dfa11641a598fcc29b2ad32afc'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:05.442869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1378/2749] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:33:05 2022] DELETE /identity/v3/users/fefdd9dfa11641a598fcc29b2ad32afc => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:33:05.612273 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d90990-7dec-46c7-b098-7ed24cdfac8f service placement] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:05.628186 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d90990-7dec-46c7-b098-7ed24cdfac8f service placement] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:05.685001 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9100d-f903-48df-a25f-88e3be39c918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:05.685869 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9100d-f903-48df-a25f-88e3be39c918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:05.686159 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9100d-f903-48df-a25f-88e3be39c918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:05.686432 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9100d-f903-48df-a25f-88e3be39c918 admin admin] PATH_INFO: `/v3/projects/4c6a6f6bdf7743cba1a36a3e8f67cf02` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:05.687656 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9100d-f903-48df-a25f-88e3be39c918 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c6a6f6bdf7743cba1a36a3e8f67cf02)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:05.689551 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9100d-f903-48df-a25f-88e3be39c918 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:05.735501 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-16a9100d-f903-48df-a25f-88e3be39c918 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c6a6f6bdf7743cba1a36a3e8f67cf02', 'request_id': 'req-16a9100d-f903-48df-a25f-88e3be39c918'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:05.742452 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-16a9100d-f903-48df-a25f-88e3be39c918 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c6a6f6bdf7743cba1a36a3e8f67cf02', 'request_id': 'req-16a9100d-f903-48df-a25f-88e3be39c918'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:05.748516 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-16a9100d-f903-48df-a25f-88e3be39c918 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c6a6f6bdf7743cba1a36a3e8f67cf02', 'request_id': 'req-16a9100d-f903-48df-a25f-88e3be39c918'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:05.759431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-16a9100d-f903-48df-a25f-88e3be39c918 admin admin] The token cache is being invalidate because project 4c6a6f6bdf7743cba1a36a3e8f67cf02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:33:05.759716 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-16a9100d-f903-48df-a25f-88e3be39c918 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:05.760517 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-16a9100d-f903-48df-a25f-88e3be39c918 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:05.762055 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1372/2750] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:33:05 2022] DELETE /identity/v3/projects/4c6a6f6bdf7743cba1a36a3e8f67cf02 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:33:05.766983 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ddca6-892a-432c-8ea0-644241a008c1 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:05.767307 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ddca6-892a-432c-8ea0-644241a008c1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:05.767590 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ddca6-892a-432c-8ea0-644241a008c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:05.785935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-606ddca6-892a-432c-8ea0-644241a008c1 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:33:05.787189 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-606ddca6-892a-432c-8ea0-644241a008c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:05.846684 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1379/2751] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:33:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:33:05.855145 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3c29c-79d9-462b-b544-e8941565d942 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:05.855468 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3c29c-79d9-462b-b544-e8941565d942 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:05.855777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3c29c-79d9-462b-b544-e8941565d942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:05.874886 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c1e3c29c-79d9-462b-b544-e8941565d942 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:33:05.876131 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c1e3c29c-79d9-462b-b544-e8941565d942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:05.883995 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1373/2752] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:33:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:33:05.888708 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ddca6-892a-432c-8ea0-644241a008c1 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:05.898378 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ddca6-892a-432c-8ea0-644241a008c1 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:05.902955 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67df90-6873-4672-ac2c-e3f8e0bd2762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:05.903631 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67df90-6873-4672-ac2c-e3f8e0bd2762 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:05.903915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67df90-6873-4672-ac2c-e3f8e0bd2762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:05.904192 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67df90-6873-4672-ac2c-e3f8e0bd2762 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:05.904926 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c67df90-6873-4672-ac2c-e3f8e0bd2762 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:33:05.905259 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c67df90-6873-4672-ac2c-e3f8e0bd2762 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:05.906836 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c67df90-6873-4672-ac2c-e3f8e0bd2762 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:05.908303 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:33:05.920558 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1380/2753] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:33:05.924501 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3c29c-79d9-462b-b544-e8941565d942 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:05.934161 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3c29c-79d9-462b-b544-e8941565d942 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:05.938771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928dc88f-704b-45fd-b3ce-37a95cfa3fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:05.939449 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dc88f-704b-45fd-b3ce-37a95cfa3fa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:05.939732 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dc88f-704b-45fd-b3ce-37a95cfa3fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:05.940005 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dc88f-704b-45fd-b3ce-37a95cfa3fa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:05.940693 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928dc88f-704b-45fd-b3ce-37a95cfa3fa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:05.942571 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928dc88f-704b-45fd-b3ce-37a95cfa3fa0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:05.969489 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1374/2754] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:33:05 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:33:05.973772 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67df90-6873-4672-ac2c-e3f8e0bd2762 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:05.984186 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67df90-6873-4672-ac2c-e3f8e0bd2762 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:06.041650 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd835b-e744-44cb-9d7d-18bae5da107d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:06.042272 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd835b-e744-44cb-9d7d-18bae5da107d admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:06.042590 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd835b-e744-44cb-9d7d-18bae5da107d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:06.043015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd835b-e744-44cb-9d7d-18bae5da107d admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:06.044184 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd835b-e744-44cb-9d7d-18bae5da107d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:06.046510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd835b-e744-44cb-9d7d-18bae5da107d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:06.065875 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1381/2755] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:33:05 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:33:06.069977 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928dc88f-704b-45fd-b3ce-37a95cfa3fa0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:06.080214 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928dc88f-704b-45fd-b3ce-37a95cfa3fa0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:06.084982 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab10c6c-316d-42b5-a655-fe56f83d623c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:06.085642 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab10c6c-316d-42b5-a655-fe56f83d623c admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:06.085928 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab10c6c-316d-42b5-a655-fe56f83d623c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:06.086206 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab10c6c-316d-42b5-a655-fe56f83d623c admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:06.086937 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab10c6c-316d-42b5-a655-fe56f83d623c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:06.089075 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab10c6c-316d-42b5-a655-fe56f83d623c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:06.099633 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1375/2756] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:33:06.104381 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd835b-e744-44cb-9d7d-18bae5da107d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:06.114319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd835b-e744-44cb-9d7d-18bae5da107d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:06.119028 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911696b-39da-41af-9584-a7214757bf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:06.119727 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911696b-39da-41af-9584-a7214757bf73 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:06.120013 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911696b-39da-41af-9584-a7214757bf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:06.120295 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911696b-39da-41af-9584-a7214757bf73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:06.121021 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911696b-39da-41af-9584-a7214757bf73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:06.122619 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911696b-39da-41af-9584-a7214757bf73 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:06.132543 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1382/2757] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:33:06.136592 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab10c6c-316d-42b5-a655-fe56f83d623c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:06.146487 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab10c6c-316d-42b5-a655-fe56f83d623c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:06.151759 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502d331-351a-4202-9e15-9f4ba7e9eded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:06.152636 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502d331-351a-4202-9e15-9f4ba7e9eded admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:06.152918 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502d331-351a-4202-9e15-9f4ba7e9eded admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:06.153193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502d331-351a-4202-9e15-9f4ba7e9eded admin admin] PATH_INFO: `/v3/projects/a68951caf41c425cb3e1335e30d52a3c/users/a52df1ab67cb42b69f333adede204a61/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:06.169256 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4502d331-351a-4202-9e15-9f4ba7e9eded admin admin] RBAC: Authorizing `identity:create_grant(project_id=a68951caf41c425cb3e1335e30d52a3c, user_id=a52df1ab67cb42b69f333adede204a61, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:06.171953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4502d331-351a-4202-9e15-9f4ba7e9eded admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:06.180485 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:33:06.185905 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1376/2758] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:33:06 2022] PUT /identity/v3/projects/a68951caf41c425cb3e1335e30d52a3c/users/a52df1ab67cb42b69f333adede204a61/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:33:06.190741 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7ee76-05c1-48b3-b47f-cf6d828196e9 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:06.191047 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7ee76-05c1-48b3-b47f-cf6d828196e9 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:06.191342 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7ee76-05c1-48b3-b47f-cf6d828196e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:06.209191 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-4be7ee76-05c1-48b3-b47f-cf6d828196e9 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:33:06.210379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4be7ee76-05c1-48b3-b47f-cf6d828196e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:06.268644 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1383/2759] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:33:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:33:06.278964 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502d331-351a-4202-9e15-9f4ba7e9eded admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:06.280682 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4502d331-351a-4202-9e15-9f4ba7e9eded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:06.336142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502d331-351a-4202-9e15-9f4ba7e9eded admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:06.340383 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4a6d8-e178-4c19-9df7-c838082614bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:06.341032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4a6d8-e178-4c19-9df7-c838082614bf service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:06.341317 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4a6d8-e178-4c19-9df7-c838082614bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:06.341597 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4a6d8-e178-4c19-9df7-c838082614bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:06.350904 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda4a6d8-e178-4c19-9df7-c838082614bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:06.352370 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda4a6d8-e178-4c19-9df7-c838082614bf service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:06.361950 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1377/2760] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:33:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:33:10.192457 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839dd449-d4c2-42ca-95d2-7468fbda0c23 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:10.193020 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839dd449-d4c2-42ca-95d2-7468fbda0c23 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:10.193020 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839dd449-d4c2-42ca-95d2-7468fbda0c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:10.222154 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-839dd449-d4c2-42ca-95d2-7468fbda0c23 None None] MFA Rules not processed for user `a52df1ab67cb42b69f333adede204a61`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:33:10.223577 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-839dd449-d4c2-42ca-95d2-7468fbda0c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:10.294256 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1384/2761] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:33:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:33:10.309633 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4a6d8-e178-4c19-9df7-c838082614bf service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:10.311101 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-eda4a6d8-e178-4c19-9df7-c838082614bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:10.389876 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4a6d8-e178-4c19-9df7-c838082614bf service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:10.395705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857fb12-b099-48fe-8c3e-3cc0db2d5aad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:10.396401 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857fb12-b099-48fe-8c3e-3cc0db2d5aad service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:10.396680 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857fb12-b099-48fe-8c3e-3cc0db2d5aad service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:10.396953 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857fb12-b099-48fe-8c3e-3cc0db2d5aad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:10.406132 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857fb12-b099-48fe-8c3e-3cc0db2d5aad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:10.407767 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857fb12-b099-48fe-8c3e-3cc0db2d5aad service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:10.417589 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1378/2762] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:33:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:33:12.762357 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91d174-0d62-4132-ba02-f01c012000a9 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:12.762854 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91d174-0d62-4132-ba02-f01c012000a9 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:12.763048 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91d174-0d62-4132-ba02-f01c012000a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:12.770302 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:33:12.785509 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-3c91d174-0d62-4132-ba02-f01c012000a9 None None] MFA Rules not processed for user `a52df1ab67cb42b69f333adede204a61`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:33:12.786714 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3c91d174-0d62-4132-ba02-f01c012000a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:12.794896 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1385/2763] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:33:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:33:13.152914 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857fb12-b099-48fe-8c3e-3cc0db2d5aad service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:13.164368 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857fb12-b099-48fe-8c3e-3cc0db2d5aad service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:13.168924 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9fab0-4b42-4655-886c-bfc8f1359f4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:13.169619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9fab0-4b42-4655-886c-bfc8f1359f4e service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:13.169911 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9fab0-4b42-4655-886c-bfc8f1359f4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:13.170193 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9fab0-4b42-4655-886c-bfc8f1359f4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:13.172015 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c5b9fab0-4b42-4655-886c-bfc8f1359f4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:13.182534 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9fab0-4b42-4655-886c-bfc8f1359f4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:13.184020 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9fab0-4b42-4655-886c-bfc8f1359f4e service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:13.193627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1379/2764] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:33:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:33:22.781109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:33:23.189544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=66 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:33:25.281558 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c91d174-0d62-4132-ba02-f01c012000a9 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:25.283016 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3c91d174-0d62-4132-ba02-f01c012000a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:25.351889 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c91d174-0d62-4132-ba02-f01c012000a9 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:25.358141 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546ce88-d711-455c-bb1b-79a5dbcccdc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:25.358801 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546ce88-d711-455c-bb1b-79a5dbcccdc5 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:25.359082 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546ce88-d711-455c-bb1b-79a5dbcccdc5 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:25.359386 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546ce88-d711-455c-bb1b-79a5dbcccdc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:25.361348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f546ce88-d711-455c-bb1b-79a5dbcccdc5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:25.374446 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f546ce88-d711-455c-bb1b-79a5dbcccdc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:25.376077 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f546ce88-d711-455c-bb1b-79a5dbcccdc5 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:25.387452 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1386/2765] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:33:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:33:35.382100 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=28 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:33:43.979492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9fab0-4b42-4655-886c-bfc8f1359f4e service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:44.019104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9fab0-4b42-4655-886c-bfc8f1359f4e service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:44.022980 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00816d-18f4-45d0-afda-b7333422621c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:44.023661 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00816d-18f4-45d0-afda-b7333422621c service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:44.023943 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00816d-18f4-45d0-afda-b7333422621c service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:44.024222 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00816d-18f4-45d0-afda-b7333422621c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:44.025961 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4a00816d-18f4-45d0-afda-b7333422621c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:44.062646 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00816d-18f4-45d0-afda-b7333422621c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:44.064103 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00816d-18f4-45d0-afda-b7333422621c service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:44.083265 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1380/2766] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:33:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:33:50.356653 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546ce88-d711-455c-bb1b-79a5dbcccdc5 service nova] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:50.358737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f546ce88-d711-455c-bb1b-79a5dbcccdc5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:50.379347 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546ce88-d711-455c-bb1b-79a5dbcccdc5 service nova] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:50.384268 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3bd12c-ad58-401b-9831-00d3d4fe2918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:50.385161 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3bd12c-ad58-401b-9831-00d3d4fe2918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:50.385441 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3bd12c-ad58-401b-9831-00d3d4fe2918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:50.385715 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3bd12c-ad58-401b-9831-00d3d4fe2918 admin admin] PATH_INFO: `/v3/users/d0b828a3dca14f2283a16894364d6eef` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:50.387005 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3bd12c-ad58-401b-9831-00d3d4fe2918 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0b828a3dca14f2283a16894364d6eef)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:50.389011 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3bd12c-ad58-401b-9831-00d3d4fe2918 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:50.435599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9b3bd12c-ad58-401b-9831-00d3d4fe2918 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0b828a3dca14f2283a16894364d6eef'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:50.447590 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9b3bd12c-ad58-401b-9831-00d3d4fe2918 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0b828a3dca14f2283a16894364d6eef'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:50.448258 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9b3bd12c-ad58-401b-9831-00d3d4fe2918 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0b828a3dca14f2283a16894364d6eef'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:50.448869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9b3bd12c-ad58-401b-9831-00d3d4fe2918 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0b828a3dca14f2283a16894364d6eef'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:50.455983 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9b3bd12c-ad58-401b-9831-00d3d4fe2918 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0b828a3dca14f2283a16894364d6eef'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:50.468552 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1387/2767] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:33:50 2022] DELETE /identity/v3/users/d0b828a3dca14f2283a16894364d6eef => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:33:50.646637 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00816d-18f4-45d0-afda-b7333422621c service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:50.658343 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00816d-18f4-45d0-afda-b7333422621c service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:50.715050 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e220ca-1ea2-4adb-8ff2-a02e563999f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:50.716171 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e220ca-1ea2-4adb-8ff2-a02e563999f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:50.716451 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e220ca-1ea2-4adb-8ff2-a02e563999f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:50.716898 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e220ca-1ea2-4adb-8ff2-a02e563999f7 admin admin] PATH_INFO: `/v3/projects/6ee52855d08e467f9d2f02685d7bf541` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:50.718169 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e220ca-1ea2-4adb-8ff2-a02e563999f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ee52855d08e467f9d2f02685d7bf541)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:50.720150 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e220ca-1ea2-4adb-8ff2-a02e563999f7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:50.771847 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-14e220ca-1ea2-4adb-8ff2-a02e563999f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ee52855d08e467f9d2f02685d7bf541', 'request_id': 'req-14e220ca-1ea2-4adb-8ff2-a02e563999f7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:50.780305 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-14e220ca-1ea2-4adb-8ff2-a02e563999f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ee52855d08e467f9d2f02685d7bf541', 'request_id': 'req-14e220ca-1ea2-4adb-8ff2-a02e563999f7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:50.786051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-14e220ca-1ea2-4adb-8ff2-a02e563999f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ee52855d08e467f9d2f02685d7bf541', 'request_id': 'req-14e220ca-1ea2-4adb-8ff2-a02e563999f7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:50.798352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-14e220ca-1ea2-4adb-8ff2-a02e563999f7 admin admin] The token cache is being invalidate because project 6ee52855d08e467f9d2f02685d7bf541 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:33:50.798635 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-14e220ca-1ea2-4adb-8ff2-a02e563999f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:50.799331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-14e220ca-1ea2-4adb-8ff2-a02e563999f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:50.800873 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1381/2768] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:33:50 2022] DELETE /identity/v3/projects/6ee52855d08e467f9d2f02685d7bf541 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:33:50.805084 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3bd12c-ad58-401b-9831-00d3d4fe2918 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:50.806701 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9b3bd12c-ad58-401b-9831-00d3d4fe2918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:50.820285 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3bd12c-ad58-401b-9831-00d3d4fe2918 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:50.878400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71ab37-3bd5-4346-aabb-b19e4737cc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:50.879199 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71ab37-3bd5-4346-aabb-b19e4737cc41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:50.879531 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71ab37-3bd5-4346-aabb-b19e4737cc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:50.879791 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71ab37-3bd5-4346-aabb-b19e4737cc41 admin admin] PATH_INFO: `/v3/users/fea498c8e7794e01a0bc5b4fb974229f` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:50.880970 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71ab37-3bd5-4346-aabb-b19e4737cc41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fea498c8e7794e01a0bc5b4fb974229f)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:50.882870 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71ab37-3bd5-4346-aabb-b19e4737cc41 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:50.929099 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9a71ab37-3bd5-4346-aabb-b19e4737cc41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fea498c8e7794e01a0bc5b4fb974229f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:50.941252 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9a71ab37-3bd5-4346-aabb-b19e4737cc41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fea498c8e7794e01a0bc5b4fb974229f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:50.941920 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9a71ab37-3bd5-4346-aabb-b19e4737cc41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fea498c8e7794e01a0bc5b4fb974229f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:50.942535 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9a71ab37-3bd5-4346-aabb-b19e4737cc41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fea498c8e7794e01a0bc5b4fb974229f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:50.949435 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9a71ab37-3bd5-4346-aabb-b19e4737cc41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fea498c8e7794e01a0bc5b4fb974229f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:50.951308 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:33:50.964022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1388/2769] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:33:50 2022] DELETE /identity/v3/users/fea498c8e7794e01a0bc5b4fb974229f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:33:51.132191 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e220ca-1ea2-4adb-8ff2-a02e563999f7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:51.133705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-14e220ca-1ea2-4adb-8ff2-a02e563999f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:51.147587 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e220ca-1ea2-4adb-8ff2-a02e563999f7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:51.167944 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:33:51.206719 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145a7f8-b26a-483b-828c-dd8e611f263a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:51.207641 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145a7f8-b26a-483b-828c-dd8e611f263a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:51.207923 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145a7f8-b26a-483b-828c-dd8e611f263a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:51.208199 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145a7f8-b26a-483b-828c-dd8e611f263a admin admin] PATH_INFO: `/v3/projects/25f284306301418aae8458f7fea5b9eb` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:51.209398 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145a7f8-b26a-483b-828c-dd8e611f263a admin admin] RBAC: Authorizing `identity:delete_project(project_id=25f284306301418aae8458f7fea5b9eb)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:51.211326 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145a7f8-b26a-483b-828c-dd8e611f263a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:51.256610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e145a7f8-b26a-483b-828c-dd8e611f263a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25f284306301418aae8458f7fea5b9eb', 'request_id': 'req-e145a7f8-b26a-483b-828c-dd8e611f263a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:51.264435 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e145a7f8-b26a-483b-828c-dd8e611f263a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25f284306301418aae8458f7fea5b9eb', 'request_id': 'req-e145a7f8-b26a-483b-828c-dd8e611f263a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:51.270089 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e145a7f8-b26a-483b-828c-dd8e611f263a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25f284306301418aae8458f7fea5b9eb', 'request_id': 'req-e145a7f8-b26a-483b-828c-dd8e611f263a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:51.284397 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e145a7f8-b26a-483b-828c-dd8e611f263a admin admin] The token cache is being invalidate because project 25f284306301418aae8458f7fea5b9eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:33:51.284688 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e145a7f8-b26a-483b-828c-dd8e611f263a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:51.285334 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e145a7f8-b26a-483b-828c-dd8e611f263a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:33:51.286885 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1382/2770] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:33:51 2022] DELETE /identity/v3/projects/25f284306301418aae8458f7fea5b9eb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:34:00.961072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:34:01.281399 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:34:10.451442 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02089a52-1337-47e4-b1d1-ab6d109ebf67 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:34:10.451939 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02089a52-1337-47e4-b1d1-ab6d109ebf67 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:34:10.452066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02089a52-1337-47e4-b1d1-ab6d109ebf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:34:10.469629 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-02089a52-1337-47e4-b1d1-ab6d109ebf67 None None] MFA Rules not processed for user `a52df1ab67cb42b69f333adede204a61`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:34:10.470819 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-02089a52-1337-47e4-b1d1-ab6d109ebf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:34:10.526217 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1389/2771] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:34:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:34:10.825567 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145a7f8-b26a-483b-828c-dd8e611f263a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:34:10.827174 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e145a7f8-b26a-483b-828c-dd8e611f263a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:34:10.894870 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145a7f8-b26a-483b-828c-dd8e611f263a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:34:10.898567 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733c4d8-22db-43c0-bfcd-d218ad9f7029 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:34:10.899235 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733c4d8-22db-43c0-bfcd-d218ad9f7029 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:34:10.899535 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733c4d8-22db-43c0-bfcd-d218ad9f7029 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:34:10.899815 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733c4d8-22db-43c0-bfcd-d218ad9f7029 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:34:10.901586 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8733c4d8-22db-43c0-bfcd-d218ad9f7029 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:34:10.911682 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8733c4d8-22db-43c0-bfcd-d218ad9f7029 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:34:10.913104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8733c4d8-22db-43c0-bfcd-d218ad9f7029 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:34:10.921584 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1383/2772] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:34:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:34:20.519558 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=25 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:34:20.917432 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=31 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:34:32.988584 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02089a52-1337-47e4-b1d1-ab6d109ebf67 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:34:32.990028 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-02089a52-1337-47e4-b1d1-ab6d109ebf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:34:33.079406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02089a52-1337-47e4-b1d1-ab6d109ebf67 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:34:33.084380 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f4cea-8584-4d8d-8696-63a487a662a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab73cf3febbf4c768cd0116306f16c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:34:33.085022 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f4cea-8584-4d8d-8696-63a487a662a8 service placement] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:34:33.085303 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f4cea-8584-4d8d-8696-63a487a662a8 service placement] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:34:33.085574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f4cea-8584-4d8d-8696-63a487a662a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:34:33.087384 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2b4f4cea-8584-4d8d-8696-63a487a662a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:34:33.099472 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f4cea-8584-4d8d-8696-63a487a662a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:34:33.101033 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f4cea-8584-4d8d-8696-63a487a662a8 service placement] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:34:33.111456 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1390/2773] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:34:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:34:43.106547 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=41 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:35:07.837049 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021fa8b-65a9-42b3-a5c0-5d4e7f9a3039 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:35:07.837582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021fa8b-65a9-42b3-a5c0-5d4e7f9a3039 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:35:07.837582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021fa8b-65a9-42b3-a5c0-5d4e7f9a3039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:35:07.855501 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3021fa8b-65a9-42b3-a5c0-5d4e7f9a3039 None None] MFA Rules not processed for user `a52df1ab67cb42b69f333adede204a61`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:35:07.856709 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3021fa8b-65a9-42b3-a5c0-5d4e7f9a3039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:35:07.864189 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1384/2774] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:35:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:35:17.850525 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:36:08.990036 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0af08-ce29-4b38-8462-63ce02ce8c9d None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:36:08.991294 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0af08-ce29-4b38-8462-63ce02ce8c9d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:36:08.991406 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0af08-ce29-4b38-8462-63ce02ce8c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:36:09.009383 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a1f0af08-ce29-4b38-8462-63ce02ce8c9d None None] MFA Rules not processed for user `a52df1ab67cb42b69f333adede204a61`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:36:09.010613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a1f0af08-ce29-4b38-8462-63ce02ce8c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:36:09.017399 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1391/2775] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:36:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:36:19.004005 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:36:55.401346 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021fa8b-65a9-42b3-a5c0-5d4e7f9a3039 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:36:55.402803 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3021fa8b-65a9-42b3-a5c0-5d4e7f9a3039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:36:55.480127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021fa8b-65a9-42b3-a5c0-5d4e7f9a3039 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:36:55.483836 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea530224-e35f-48ef-ac22-6de71dcfdf00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:36:55.484477 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea530224-e35f-48ef-ac22-6de71dcfdf00 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:36:55.484757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea530224-e35f-48ef-ac22-6de71dcfdf00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:36:55.485032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea530224-e35f-48ef-ac22-6de71dcfdf00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:36:55.486759 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ea530224-e35f-48ef-ac22-6de71dcfdf00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:36:55.575969 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea530224-e35f-48ef-ac22-6de71dcfdf00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:36:55.577600 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea530224-e35f-48ef-ac22-6de71dcfdf00 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:36:55.598414 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1385/2776] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:36:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:37:02.377820 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0af08-ce29-4b38-8462-63ce02ce8c9d None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:02.387574 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0af08-ce29-4b38-8462-63ce02ce8c9d None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:02.391106 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be815fa-7243-4a9b-8634-010689b1df81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:02.391782 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be815fa-7243-4a9b-8634-010689b1df81 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:02.392069 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be815fa-7243-4a9b-8634-010689b1df81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:02.392348 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be815fa-7243-4a9b-8634-010689b1df81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:02.394110 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1be815fa-7243-4a9b-8634-010689b1df81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:02.407490 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be815fa-7243-4a9b-8634-010689b1df81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:02.408945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be815fa-7243-4a9b-8634-010689b1df81 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:02.420399 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1392/2777] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:37:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:37:05.595869 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=75 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:06.154831 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea530224-e35f-48ef-ac22-6de71dcfdf00 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:06.156466 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ea530224-e35f-48ef-ac22-6de71dcfdf00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:06.173322 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea530224-e35f-48ef-ac22-6de71dcfdf00 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:06.179853 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acade367-0fd9-43dd-8c50-ea4982baa0e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae726fbc9cf40e68c7587552a118a7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:06.180629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acade367-0fd9-43dd-8c50-ea4982baa0e2 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:06.180811 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acade367-0fd9-43dd-8c50-ea4982baa0e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:06.181086 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acade367-0fd9-43dd-8c50-ea4982baa0e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:06.183482 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-acade367-0fd9-43dd-8c50-ea4982baa0e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:06.197772 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acade367-0fd9-43dd-8c50-ea4982baa0e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:06.200105 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acade367-0fd9-43dd-8c50-ea4982baa0e2 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:06.214158 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1386/2778] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:37:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:37:12.415587 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=6 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:14.508887 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be815fa-7243-4a9b-8634-010689b1df81 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:14.510454 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1be815fa-7243-4a9b-8634-010689b1df81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:14.590822 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be815fa-7243-4a9b-8634-010689b1df81 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:14.595362 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f5c1c-bd1a-4160-b1a6-12b544d745f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:14.596220 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f5c1c-bd1a-4160-b1a6-12b544d745f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:14.596502 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f5c1c-bd1a-4160-b1a6-12b544d745f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:14.596776 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f5c1c-bd1a-4160-b1a6-12b544d745f9 admin admin] PATH_INFO: `/v3/users/a52df1ab67cb42b69f333adede204a61` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:14.597941 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f5c1c-bd1a-4160-b1a6-12b544d745f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a52df1ab67cb42b69f333adede204a61)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:14.599881 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f5c1c-bd1a-4160-b1a6-12b544d745f9 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:14.647471 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ac7f5c1c-bd1a-4160-b1a6-12b544d745f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a52df1ab67cb42b69f333adede204a61'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:14.659853 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ac7f5c1c-bd1a-4160-b1a6-12b544d745f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a52df1ab67cb42b69f333adede204a61'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:14.660583 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ac7f5c1c-bd1a-4160-b1a6-12b544d745f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a52df1ab67cb42b69f333adede204a61'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:14.661147 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ac7f5c1c-bd1a-4160-b1a6-12b544d745f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a52df1ab67cb42b69f333adede204a61'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:14.668089 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ac7f5c1c-bd1a-4160-b1a6-12b544d745f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a52df1ab67cb42b69f333adede204a61'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:14.681202 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1393/2779] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:37:14 2022] DELETE /identity/v3/users/a52df1ab67cb42b69f333adede204a61 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:14.856108 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acade367-0fd9-43dd-8c50-ea4982baa0e2 service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:14.866829 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acade367-0fd9-43dd-8c50-ea4982baa0e2 service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:14.929735 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c1b96-d7ee-476d-b05e-7db65210d7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:14.930600 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c1b96-d7ee-476d-b05e-7db65210d7ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:14.930882 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c1b96-d7ee-476d-b05e-7db65210d7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:14.931159 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c1b96-d7ee-476d-b05e-7db65210d7ca admin admin] PATH_INFO: `/v3/projects/a68951caf41c425cb3e1335e30d52a3c` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:14.932385 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c1b96-d7ee-476d-b05e-7db65210d7ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=a68951caf41c425cb3e1335e30d52a3c)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:14.934550 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c1b96-d7ee-476d-b05e-7db65210d7ca admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:14.983083 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8e1c1b96-d7ee-476d-b05e-7db65210d7ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a68951caf41c425cb3e1335e30d52a3c', 'request_id': 'req-8e1c1b96-d7ee-476d-b05e-7db65210d7ca'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:14.990514 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8e1c1b96-d7ee-476d-b05e-7db65210d7ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a68951caf41c425cb3e1335e30d52a3c', 'request_id': 'req-8e1c1b96-d7ee-476d-b05e-7db65210d7ca'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:14.996649 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8e1c1b96-d7ee-476d-b05e-7db65210d7ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a68951caf41c425cb3e1335e30d52a3c', 'request_id': 'req-8e1c1b96-d7ee-476d-b05e-7db65210d7ca'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:15.007416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8e1c1b96-d7ee-476d-b05e-7db65210d7ca admin admin] The token cache is being invalidate because project a68951caf41c425cb3e1335e30d52a3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:37:15.007691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8e1c1b96-d7ee-476d-b05e-7db65210d7ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:15.008339 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8e1c1b96-d7ee-476d-b05e-7db65210d7ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:15.009876 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1387/2780] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:37:14 2022] DELETE /identity/v3/projects/a68951caf41c425cb3e1335e30d52a3c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:15.014924 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4bf5e5-ce18-4e82-b901-e7e0ad573be2 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:15.015294 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4bf5e5-ce18-4e82-b901-e7e0ad573be2 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:15.015577 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4bf5e5-ce18-4e82-b901-e7e0ad573be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:15.071735 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-8f4bf5e5-ce18-4e82-b901-e7e0ad573be2 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:15.072990 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8f4bf5e5-ce18-4e82-b901-e7e0ad573be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:15.116004 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:15.135402 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1394/2781] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:15.191606 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1eb89-9aa7-414a-a58d-449041682077 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:15.192060 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1eb89-9aa7-414a-a58d-449041682077 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:15.192224 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1eb89-9aa7-414a-a58d-449041682077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:15.211387 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-fbf1eb89-9aa7-414a-a58d-449041682077 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:15.212587 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fbf1eb89-9aa7-414a-a58d-449041682077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:15.220610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1388/2782] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:15.224829 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4bf5e5-ce18-4e82-b901-e7e0ad573be2 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:15.235496 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4bf5e5-ce18-4e82-b901-e7e0ad573be2 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:15.240206 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ce1d7-3b1e-4f28-a232-b09d0e79e64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:15.240863 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ce1d7-3b1e-4f28-a232-b09d0e79e64c admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:15.241155 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ce1d7-3b1e-4f28-a232-b09d0e79e64c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:15.241434 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ce1d7-3b1e-4f28-a232-b09d0e79e64c admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:15.242196 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ce1d7-3b1e-4f28-a232-b09d0e79e64c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:37:15.242511 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ce1d7-3b1e-4f28-a232-b09d0e79e64c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:15.244140 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ce1d7-3b1e-4f28-a232-b09d0e79e64c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:15.255666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1395/2783] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:37:15.259894 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1eb89-9aa7-414a-a58d-449041682077 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:15.268941 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1eb89-9aa7-414a-a58d-449041682077 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:15.274337 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0c7f2-2184-4b2e-8d66-99fce67bae3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:15.275023 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0c7f2-2184-4b2e-8d66-99fce67bae3c admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:15.275338 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0c7f2-2184-4b2e-8d66-99fce67bae3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:15.275614 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0c7f2-2184-4b2e-8d66-99fce67bae3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:15.276291 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f0c7f2-2184-4b2e-8d66-99fce67bae3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:15.278192 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f0c7f2-2184-4b2e-8d66-99fce67bae3c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:15.303418 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1389/2784] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:37:15 2022] POST /identity/v3/projects => generated 363 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:37:15.308157 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ce1d7-3b1e-4f28-a232-b09d0e79e64c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:15.318134 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ce1d7-3b1e-4f28-a232-b09d0e79e64c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:15.376367 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215004d-8c88-41ce-901f-b57034fb3034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:15.377037 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215004d-8c88-41ce-901f-b57034fb3034 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:15.377322 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215004d-8c88-41ce-901f-b57034fb3034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:15.377599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215004d-8c88-41ce-901f-b57034fb3034 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:15.378300 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f215004d-8c88-41ce-901f-b57034fb3034 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:15.380247 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f215004d-8c88-41ce-901f-b57034fb3034 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:15.401360 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1396/2785] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:37:15 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:37:15.405578 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0c7f2-2184-4b2e-8d66-99fce67bae3c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:15.415887 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0c7f2-2184-4b2e-8d66-99fce67bae3c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:15.420541 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f0484-a3fe-462e-ad6d-53c8c97bb4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:15.421143 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f0484-a3fe-462e-ad6d-53c8c97bb4a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:15.421428 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f0484-a3fe-462e-ad6d-53c8c97bb4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:15.421711 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f0484-a3fe-462e-ad6d-53c8c97bb4a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:15.422467 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2f0484-a3fe-462e-ad6d-53c8c97bb4a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:15.424098 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2f0484-a3fe-462e-ad6d-53c8c97bb4a9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:15.434866 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1390/2786] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:37:15.438889 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215004d-8c88-41ce-901f-b57034fb3034 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:15.450469 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215004d-8c88-41ce-901f-b57034fb3034 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:15.455589 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb2473-ed2d-434f-9ee5-1e638147a4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:15.456260 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb2473-ed2d-434f-9ee5-1e638147a4b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:15.456544 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb2473-ed2d-434f-9ee5-1e638147a4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:15.456827 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb2473-ed2d-434f-9ee5-1e638147a4b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:15.457551 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb2473-ed2d-434f-9ee5-1e638147a4b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:15.459131 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb2473-ed2d-434f-9ee5-1e638147a4b5 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:15.473283 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1397/2787] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:37:15.474458 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f0484-a3fe-462e-ad6d-53c8c97bb4a9 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:15.484678 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f0484-a3fe-462e-ad6d-53c8c97bb4a9 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:15.489307 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3bc28-4b3c-4f7a-85e1-70b0af583417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:15.490181 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3bc28-4b3c-4f7a-85e1-70b0af583417 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:15.490464 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3bc28-4b3c-4f7a-85e1-70b0af583417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:15.490737 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3bc28-4b3c-4f7a-85e1-70b0af583417 admin admin] PATH_INFO: `/v3/projects/896e41e087fe425b95edfabd61b33c75/users/aff267acf51a44c6a8386df916d3f7b2/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:15.505497 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3bc28-4b3c-4f7a-85e1-70b0af583417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=896e41e087fe425b95edfabd61b33c75, user_id=aff267acf51a44c6a8386df916d3f7b2, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:15.508828 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3bc28-4b3c-4f7a-85e1-70b0af583417 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:15.521506 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1391/2788] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:37:15 2022] PUT /identity/v3/projects/896e41e087fe425b95edfabd61b33c75/users/aff267acf51a44c6a8386df916d3f7b2/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:37:15.532033 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5333099-0157-4c46-a090-b46398b43f8a None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:15.532532 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5333099-0157-4c46-a090-b46398b43f8a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:15.532642 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5333099-0157-4c46-a090-b46398b43f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:15.559980 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-b5333099-0157-4c46-a090-b46398b43f8a None None] MFA Rules not processed for user `aff267acf51a44c6a8386df916d3f7b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:15.561191 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b5333099-0157-4c46-a090-b46398b43f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:15.625133 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1398/2789] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:15 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:15.629708 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3bc28-4b3c-4f7a-85e1-70b0af583417 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:15.643657 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:15.692793 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3bc28-4b3c-4f7a-85e1-70b0af583417 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:15.698068 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9ea9c-e7f5-472a-9201-fade166bc06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:15.698718 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9ea9c-e7f5-472a-9201-fade166bc06f admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:15.698998 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9ea9c-e7f5-472a-9201-fade166bc06f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:15.699299 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9ea9c-e7f5-472a-9201-fade166bc06f admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:15.700013 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9ea9c-e7f5-472a-9201-fade166bc06f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:15.701889 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9ea9c-e7f5-472a-9201-fade166bc06f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:15.728215 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1392/2790] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:37:15 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:37:15.732488 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5333099-0157-4c46-a090-b46398b43f8a None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:15.743028 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5333099-0157-4c46-a090-b46398b43f8a None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:15.784064 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:15.799652 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf79cfb-ea0a-4379-b09e-317f22d510f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:15.800314 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf79cfb-ea0a-4379-b09e-317f22d510f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:15.800594 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf79cfb-ea0a-4379-b09e-317f22d510f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:15.800870 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf79cfb-ea0a-4379-b09e-317f22d510f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:15.801555 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf79cfb-ea0a-4379-b09e-317f22d510f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:15.803479 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf79cfb-ea0a-4379-b09e-317f22d510f0 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:15.823313 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1399/2791] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:37:15 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:37:15.827493 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9ea9c-e7f5-472a-9201-fade166bc06f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:15.838532 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9ea9c-e7f5-472a-9201-fade166bc06f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:15.843128 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d013147-895d-4a90-ae85-8ee01f2f364a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:15.843814 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d013147-895d-4a90-ae85-8ee01f2f364a admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:15.844095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d013147-895d-4a90-ae85-8ee01f2f364a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:15.844371 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d013147-895d-4a90-ae85-8ee01f2f364a admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:15.845086 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d013147-895d-4a90-ae85-8ee01f2f364a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:15.846650 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d013147-895d-4a90-ae85-8ee01f2f364a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:15.857800 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1393/2792] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:37:15.861882 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf79cfb-ea0a-4379-b09e-317f22d510f0 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:15.871916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf79cfb-ea0a-4379-b09e-317f22d510f0 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:15.876518 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6ac4d-e3b7-4777-9c6e-bca586db6adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:15.877391 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6ac4d-e3b7-4777-9c6e-bca586db6adc admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:15.877678 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6ac4d-e3b7-4777-9c6e-bca586db6adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:15.877958 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6ac4d-e3b7-4777-9c6e-bca586db6adc admin admin] PATH_INFO: `/v3/projects/bdc1fa60b9e840e7b97f532f12a24709/users/e6fd2d808f84484f85f80aae4afe7ab1/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:15.894222 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6ac4d-e3b7-4777-9c6e-bca586db6adc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc1fa60b9e840e7b97f532f12a24709, user_id=e6fd2d808f84484f85f80aae4afe7ab1, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:15.896905 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6ac4d-e3b7-4777-9c6e-bca586db6adc admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:15.910447 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1400/2793] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:37:15 2022] PUT /identity/v3/projects/bdc1fa60b9e840e7b97f532f12a24709/users/e6fd2d808f84484f85f80aae4afe7ab1/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:37:15.914896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d013147-895d-4a90-ae85-8ee01f2f364a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:15.924732 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d013147-895d-4a90-ae85-8ee01f2f364a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:15.981659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c61c253-6e74-4a29-a356-9095e8e6c472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:15.982325 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61c253-6e74-4a29-a356-9095e8e6c472 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:15.982605 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61c253-6e74-4a29-a356-9095e8e6c472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:15.982880 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61c253-6e74-4a29-a356-9095e8e6c472 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:15.983631 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c61c253-6e74-4a29-a356-9095e8e6c472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:15.985201 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c61c253-6e74-4a29-a356-9095e8e6c472 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:15.994765 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1394/2794] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:37:15.998871 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6ac4d-e3b7-4777-9c6e-bca586db6adc admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:16.009110 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6ac4d-e3b7-4777-9c6e-bca586db6adc admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:16.013721 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acdef0-fd32-4fc2-ba87-d36d4755ba9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:16.014589 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acdef0-fd32-4fc2-ba87-d36d4755ba9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:16.014869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acdef0-fd32-4fc2-ba87-d36d4755ba9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:16.015144 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acdef0-fd32-4fc2-ba87-d36d4755ba9e admin admin] PATH_INFO: `/v3/projects/bdc1fa60b9e840e7b97f532f12a24709/users/e6fd2d808f84484f85f80aae4afe7ab1/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:16.017114 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3acdef0-fd32-4fc2-ba87-d36d4755ba9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc1fa60b9e840e7b97f532f12a24709, user_id=e6fd2d808f84484f85f80aae4afe7ab1, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:16.019777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3acdef0-fd32-4fc2-ba87-d36d4755ba9e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:16.030554 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1401/2795] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:37:15 2022] PUT /identity/v3/projects/bdc1fa60b9e840e7b97f532f12a24709/users/e6fd2d808f84484f85f80aae4afe7ab1/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:16.041716 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2474ec-3e0c-4b4d-9146-e93584bb7d18 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:16.042074 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2474ec-3e0c-4b4d-9146-e93584bb7d18 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:16.042492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2474ec-3e0c-4b4d-9146-e93584bb7d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:16.068288 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4b2474ec-3e0c-4b4d-9146-e93584bb7d18 None None] MFA Rules not processed for user `e6fd2d808f84484f85f80aae4afe7ab1`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:16.069481 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4b2474ec-3e0c-4b4d-9146-e93584bb7d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:16.137408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1395/2796] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:16 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:16.149043 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acdef0-fd32-4fc2-ba87-d36d4755ba9e admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:16.150729 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c3acdef0-fd32-4fc2-ba87-d36d4755ba9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:16.208142 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acdef0-fd32-4fc2-ba87-d36d4755ba9e admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:16.211859 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44df98b-6e62-4f35-b8a6-a13ec5f13da5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:16.212508 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44df98b-6e62-4f35-b8a6-a13ec5f13da5 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:16.212792 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44df98b-6e62-4f35-b8a6-a13ec5f13da5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:16.213066 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44df98b-6e62-4f35-b8a6-a13ec5f13da5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:16.222297 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44df98b-6e62-4f35-b8a6-a13ec5f13da5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:16.223777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44df98b-6e62-4f35-b8a6-a13ec5f13da5 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:16.233655 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1402/2797] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:37:16 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:37:16.636154 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2474ec-3e0c-4b4d-9146-e93584bb7d18 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:16.645744 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:16.699619 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2474ec-3e0c-4b4d-9146-e93584bb7d18 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:16.704104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f6987-f54b-4d72-9eb9-f6f09037a0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:16.704972 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f6987-f54b-4d72-9eb9-f6f09037a0e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:16.705253 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f6987-f54b-4d72-9eb9-f6f09037a0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:16.705529 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f6987-f54b-4d72-9eb9-f6f09037a0e8 admin admin] PATH_INFO: `/v3/users/aff267acf51a44c6a8386df916d3f7b2` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:16.706707 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f6987-f54b-4d72-9eb9-f6f09037a0e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aff267acf51a44c6a8386df916d3f7b2)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:16.708740 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f6987-f54b-4d72-9eb9-f6f09037a0e8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:16.754686 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fc9f6987-f54b-4d72-9eb9-f6f09037a0e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aff267acf51a44c6a8386df916d3f7b2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:16.755296 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fc9f6987-f54b-4d72-9eb9-f6f09037a0e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aff267acf51a44c6a8386df916d3f7b2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:16.755902 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fc9f6987-f54b-4d72-9eb9-f6f09037a0e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aff267acf51a44c6a8386df916d3f7b2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:16.763377 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fc9f6987-f54b-4d72-9eb9-f6f09037a0e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aff267acf51a44c6a8386df916d3f7b2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:16.775583 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fc9f6987-f54b-4d72-9eb9-f6f09037a0e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aff267acf51a44c6a8386df916d3f7b2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:16.788138 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1396/2798] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:37:16 2022] DELETE /identity/v3/users/aff267acf51a44c6a8386df916d3f7b2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:16.793006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f59d9-7448-432d-858c-b45eda2a7de7 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:16.793474 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f59d9-7448-432d-858c-b45eda2a7de7 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:16.793591 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f59d9-7448-432d-858c-b45eda2a7de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:16.811951 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-1d8f59d9-7448-432d-858c-b45eda2a7de7 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:16.813134 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1d8f59d9-7448-432d-858c-b45eda2a7de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:16.871800 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1403/2799] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:16.881260 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f6987-f54b-4d72-9eb9-f6f09037a0e8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:16.882919 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fc9f6987-f54b-4d72-9eb9-f6f09037a0e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:16.940459 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f6987-f54b-4d72-9eb9-f6f09037a0e8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:16.944086 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7889de16-bf77-4a03-8927-e7c9ccf01115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:16.944777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889de16-bf77-4a03-8927-e7c9ccf01115 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:16.945116 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889de16-bf77-4a03-8927-e7c9ccf01115 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:16.945416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889de16-bf77-4a03-8927-e7c9ccf01115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:16.954341 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7889de16-bf77-4a03-8927-e7c9ccf01115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:16.955821 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7889de16-bf77-4a03-8927-e7c9ccf01115 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:16.965884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1397/2800] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:37:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:37:17.239666 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f59d9-7448-432d-858c-b45eda2a7de7 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:17.241352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1d8f59d9-7448-432d-858c-b45eda2a7de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:17.253914 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f59d9-7448-432d-858c-b45eda2a7de7 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:17.259172 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d44b4-fd96-4f8b-8f02-65023f80fb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:17.260099 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d44b4-fd96-4f8b-8f02-65023f80fb5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:17.260378 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d44b4-fd96-4f8b-8f02-65023f80fb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:17.260648 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d44b4-fd96-4f8b-8f02-65023f80fb5d admin admin] PATH_INFO: `/v3/projects/896e41e087fe425b95edfabd61b33c75` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:17.261861 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d44b4-fd96-4f8b-8f02-65023f80fb5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=896e41e087fe425b95edfabd61b33c75)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:17.263796 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d44b4-fd96-4f8b-8f02-65023f80fb5d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:17.304147 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:17.316898 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-384d44b4-fd96-4f8b-8f02-65023f80fb5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '896e41e087fe425b95edfabd61b33c75', 'request_id': 'req-384d44b4-fd96-4f8b-8f02-65023f80fb5d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:17.324194 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-384d44b4-fd96-4f8b-8f02-65023f80fb5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '896e41e087fe425b95edfabd61b33c75', 'request_id': 'req-384d44b4-fd96-4f8b-8f02-65023f80fb5d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:17.330503 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-384d44b4-fd96-4f8b-8f02-65023f80fb5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '896e41e087fe425b95edfabd61b33c75', 'request_id': 'req-384d44b4-fd96-4f8b-8f02-65023f80fb5d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:17.341061 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-384d44b4-fd96-4f8b-8f02-65023f80fb5d admin admin] The token cache is being invalidate because project 896e41e087fe425b95edfabd61b33c75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:37:17.341345 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-384d44b4-fd96-4f8b-8f02-65023f80fb5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:17.341990 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-384d44b4-fd96-4f8b-8f02-65023f80fb5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:17.343570 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1404/2801] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:37:17 2022] DELETE /identity/v3/projects/896e41e087fe425b95edfabd61b33c75 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:17.347521 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7889de16-bf77-4a03-8927-e7c9ccf01115 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:17.357875 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7889de16-bf77-4a03-8927-e7c9ccf01115 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:17.387437 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:17.421121 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ea610-6315-4c0c-925a-3a5ad199b2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:17.424175 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ea610-6315-4c0c-925a-3a5ad199b2a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:17.424458 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ea610-6315-4c0c-925a-3a5ad199b2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:17.424733 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ea610-6315-4c0c-925a-3a5ad199b2a0 admin admin] PATH_INFO: `/v3/users/e6fd2d808f84484f85f80aae4afe7ab1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:17.425940 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9ea610-6315-4c0c-925a-3a5ad199b2a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6fd2d808f84484f85f80aae4afe7ab1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:17.427906 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9ea610-6315-4c0c-925a-3a5ad199b2a0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:17.473365 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2c9ea610-6315-4c0c-925a-3a5ad199b2a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6fd2d808f84484f85f80aae4afe7ab1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:17.474012 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2c9ea610-6315-4c0c-925a-3a5ad199b2a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6fd2d808f84484f85f80aae4afe7ab1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:17.474611 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2c9ea610-6315-4c0c-925a-3a5ad199b2a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6fd2d808f84484f85f80aae4afe7ab1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:17.482026 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2c9ea610-6315-4c0c-925a-3a5ad199b2a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6fd2d808f84484f85f80aae4afe7ab1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:17.494380 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2c9ea610-6315-4c0c-925a-3a5ad199b2a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6fd2d808f84484f85f80aae4afe7ab1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:17.506834 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1398/2802] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:37:17 2022] DELETE /identity/v3/users/e6fd2d808f84484f85f80aae4afe7ab1 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:17.784152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d44b4-fd96-4f8b-8f02-65023f80fb5d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:17.785590 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-384d44b4-fd96-4f8b-8f02-65023f80fb5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:17.800634 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d44b4-fd96-4f8b-8f02-65023f80fb5d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:17.865250 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768180e1-612c-4c73-b595-bdb7c313664b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:17.866034 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768180e1-612c-4c73-b595-bdb7c313664b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:17.866319 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768180e1-612c-4c73-b595-bdb7c313664b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:17.866603 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768180e1-612c-4c73-b595-bdb7c313664b admin admin] PATH_INFO: `/v3/projects/bdc1fa60b9e840e7b97f532f12a24709` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:17.868610 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768180e1-612c-4c73-b595-bdb7c313664b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdc1fa60b9e840e7b97f532f12a24709)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:17.870522 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768180e1-612c-4c73-b595-bdb7c313664b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:17.915782 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-768180e1-612c-4c73-b595-bdb7c313664b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdc1fa60b9e840e7b97f532f12a24709', 'request_id': 'req-768180e1-612c-4c73-b595-bdb7c313664b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:17.923012 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-768180e1-612c-4c73-b595-bdb7c313664b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdc1fa60b9e840e7b97f532f12a24709', 'request_id': 'req-768180e1-612c-4c73-b595-bdb7c313664b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:17.929171 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-768180e1-612c-4c73-b595-bdb7c313664b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdc1fa60b9e840e7b97f532f12a24709', 'request_id': 'req-768180e1-612c-4c73-b595-bdb7c313664b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:17.939779 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-768180e1-612c-4c73-b595-bdb7c313664b admin admin] The token cache is being invalidate because project bdc1fa60b9e840e7b97f532f12a24709 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:37:17.940058 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-768180e1-612c-4c73-b595-bdb7c313664b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:17.940693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-768180e1-612c-4c73-b595-bdb7c313664b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:17.942229 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1405/2803] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:37:17 2022] DELETE /identity/v3/projects/bdc1fa60b9e840e7b97f532f12a24709 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:17.947897 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dca265-5eae-4096-8706-10234cbcf524 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:17.948390 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dca265-5eae-4096-8706-10234cbcf524 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:17.948511 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dca265-5eae-4096-8706-10234cbcf524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:17.966425 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-25dca265-5eae-4096-8706-10234cbcf524 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:17.967669 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-25dca265-5eae-4096-8706-10234cbcf524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:18.027486 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1399/2804] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:18.035390 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5fba27-d1e6-4c32-b98d-cf4e1830f112 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:18.035684 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5fba27-d1e6-4c32-b98d-cf4e1830f112 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:18.035965 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5fba27-d1e6-4c32-b98d-cf4e1830f112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:18.054479 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-4d5fba27-d1e6-4c32-b98d-cf4e1830f112 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:18.055874 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4d5fba27-d1e6-4c32-b98d-cf4e1830f112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:18.064502 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1406/2805] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:18.070697 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dca265-5eae-4096-8706-10234cbcf524 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:18.081427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dca265-5eae-4096-8706-10234cbcf524 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:18.087376 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c414d-576c-42e7-ba16-bc0ff1ad2a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:18.088249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c414d-576c-42e7-ba16-bc0ff1ad2a17 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:18.088583 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c414d-576c-42e7-ba16-bc0ff1ad2a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:18.088942 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c414d-576c-42e7-ba16-bc0ff1ad2a17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:18.089821 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c414d-576c-42e7-ba16-bc0ff1ad2a17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:37:18.090130 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c414d-576c-42e7-ba16-bc0ff1ad2a17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:18.091749 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c414d-576c-42e7-ba16-bc0ff1ad2a17 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:18.103875 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1400/2806] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:37:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:37:18.108274 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5fba27-d1e6-4c32-b98d-cf4e1830f112 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:18.117628 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5fba27-d1e6-4c32-b98d-cf4e1830f112 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:18.122266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af9d2d-00bb-40a9-b154-c5fa3b992e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:18.122920 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af9d2d-00bb-40a9-b154-c5fa3b992e64 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:18.123205 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af9d2d-00bb-40a9-b154-c5fa3b992e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:18.123512 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af9d2d-00bb-40a9-b154-c5fa3b992e64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:18.124203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af9d2d-00bb-40a9-b154-c5fa3b992e64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:18.126101 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af9d2d-00bb-40a9-b154-c5fa3b992e64 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:18.156597 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1407/2807] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:37:18 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:37:18.158031 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c414d-576c-42e7-ba16-bc0ff1ad2a17 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:18.168888 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c414d-576c-42e7-ba16-bc0ff1ad2a17 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:18.201891 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:18.227819 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e563bf-cc24-463b-8a2d-3ac130311da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:18.228496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e563bf-cc24-463b-8a2d-3ac130311da4 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:18.228779 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e563bf-cc24-463b-8a2d-3ac130311da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:18.229057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e563bf-cc24-463b-8a2d-3ac130311da4 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:18.229863 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e563bf-cc24-463b-8a2d-3ac130311da4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:18.231780 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e563bf-cc24-463b-8a2d-3ac130311da4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:18.251249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1401/2808] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:37:18 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:37:18.255422 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af9d2d-00bb-40a9-b154-c5fa3b992e64 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:18.265699 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af9d2d-00bb-40a9-b154-c5fa3b992e64 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:18.270336 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312ea51-f7dd-431e-8f6e-262fc39229a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:18.271025 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312ea51-f7dd-431e-8f6e-262fc39229a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:18.271339 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312ea51-f7dd-431e-8f6e-262fc39229a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:18.271620 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312ea51-f7dd-431e-8f6e-262fc39229a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:18.272351 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3312ea51-f7dd-431e-8f6e-262fc39229a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:18.274556 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3312ea51-f7dd-431e-8f6e-262fc39229a6 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:18.284934 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1408/2809] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:37:18.289318 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e563bf-cc24-463b-8a2d-3ac130311da4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:18.299047 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e563bf-cc24-463b-8a2d-3ac130311da4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:18.303841 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc70c41-cd25-45ce-80fc-8d41b52ca731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:18.304757 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc70c41-cd25-45ce-80fc-8d41b52ca731 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:18.305070 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc70c41-cd25-45ce-80fc-8d41b52ca731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:18.305347 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc70c41-cd25-45ce-80fc-8d41b52ca731 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:18.306075 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc70c41-cd25-45ce-80fc-8d41b52ca731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:18.307826 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc70c41-cd25-45ce-80fc-8d41b52ca731 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:18.317650 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1402/2810] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:37:18.321672 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312ea51-f7dd-431e-8f6e-262fc39229a6 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:18.331735 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312ea51-f7dd-431e-8f6e-262fc39229a6 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:18.337266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99d999-910c-490d-bc5e-e38ef3778cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:18.338144 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99d999-910c-490d-bc5e-e38ef3778cb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:18.338429 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99d999-910c-490d-bc5e-e38ef3778cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:18.338725 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99d999-910c-490d-bc5e-e38ef3778cb2 admin admin] PATH_INFO: `/v3/projects/7d2c0ee4ef5b40b7977ce411011938b7/users/59837924d4fe4e6aae6f3ac821fd9487/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:18.349870 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:18.355710 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99d999-910c-490d-bc5e-e38ef3778cb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d2c0ee4ef5b40b7977ce411011938b7, user_id=59837924d4fe4e6aae6f3ac821fd9487, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:18.358535 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99d999-910c-490d-bc5e-e38ef3778cb2 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:18.372474 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1409/2811] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:37:18 2022] PUT /identity/v3/projects/7d2c0ee4ef5b40b7977ce411011938b7/users/59837924d4fe4e6aae6f3ac821fd9487/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:18.382837 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102a6cc-e216-4a6d-a1e3-0e1eca6eb7ef None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:18.383153 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102a6cc-e216-4a6d-a1e3-0e1eca6eb7ef None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:18.383469 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102a6cc-e216-4a6d-a1e3-0e1eca6eb7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:18.409732 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d102a6cc-e216-4a6d-a1e3-0e1eca6eb7ef None None] MFA Rules not processed for user `59837924d4fe4e6aae6f3ac821fd9487`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:18.410976 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d102a6cc-e216-4a6d-a1e3-0e1eca6eb7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:18.476678 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1403/2812] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:18 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:18.481368 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99d999-910c-490d-bc5e-e38ef3778cb2 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:18.541852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99d999-910c-490d-bc5e-e38ef3778cb2 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:18.546388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e82ad-b517-4466-8081-e4996bb16ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:18.547034 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e82ad-b517-4466-8081-e4996bb16ce1 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:18.547346 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e82ad-b517-4466-8081-e4996bb16ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:18.547622 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e82ad-b517-4466-8081-e4996bb16ce1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:18.548298 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e82ad-b517-4466-8081-e4996bb16ce1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:18.550180 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e82ad-b517-4466-8081-e4996bb16ce1 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:18.576178 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1410/2813] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:37:18 2022] POST /identity/v3/projects => generated 365 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:37:18.580300 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102a6cc-e216-4a6d-a1e3-0e1eca6eb7ef None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:18.590777 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102a6cc-e216-4a6d-a1e3-0e1eca6eb7ef None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:18.647792 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9ff82-0422-4887-974b-23f97e37d746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:18.648809 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9ff82-0422-4887-974b-23f97e37d746 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:18.649484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9ff82-0422-4887-974b-23f97e37d746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:18.649484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9ff82-0422-4887-974b-23f97e37d746 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:18.650180 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9ff82-0422-4887-974b-23f97e37d746 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:18.652244 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9ff82-0422-4887-974b-23f97e37d746 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:18.672642 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1404/2814] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:37:18 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:37:18.676677 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e82ad-b517-4466-8081-e4996bb16ce1 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:18.687484 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e82ad-b517-4466-8081-e4996bb16ce1 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:18.692051 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a86fdbf-18b9-4ddc-9fd5-8d69fa01a9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:18.692707 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86fdbf-18b9-4ddc-9fd5-8d69fa01a9c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:18.692990 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86fdbf-18b9-4ddc-9fd5-8d69fa01a9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:18.693266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86fdbf-18b9-4ddc-9fd5-8d69fa01a9c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:18.693985 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a86fdbf-18b9-4ddc-9fd5-8d69fa01a9c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:18.695646 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a86fdbf-18b9-4ddc-9fd5-8d69fa01a9c1 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:18.706933 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1411/2815] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:37:18.711072 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9ff82-0422-4887-974b-23f97e37d746 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:18.720739 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9ff82-0422-4887-974b-23f97e37d746 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:18.725188 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35bed8-f2c6-4368-9abf-dc83b30e5805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:18.727030 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35bed8-f2c6-4368-9abf-dc83b30e5805 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:18.727420 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35bed8-f2c6-4368-9abf-dc83b30e5805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:18.727700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35bed8-f2c6-4368-9abf-dc83b30e5805 admin admin] PATH_INFO: `/v3/projects/25bd5415483a437ca3aa471e34adc661/users/e4e9fc456a7349fcbce9cdcfc616a9b9/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:18.742999 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35bed8-f2c6-4368-9abf-dc83b30e5805 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25bd5415483a437ca3aa471e34adc661, user_id=e4e9fc456a7349fcbce9cdcfc616a9b9, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:18.745793 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35bed8-f2c6-4368-9abf-dc83b30e5805 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:18.758314 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1405/2816] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:37:18 2022] PUT /identity/v3/projects/25bd5415483a437ca3aa471e34adc661/users/e4e9fc456a7349fcbce9cdcfc616a9b9/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:18.762231 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a86fdbf-18b9-4ddc-9fd5-8d69fa01a9c1 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:18.771914 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a86fdbf-18b9-4ddc-9fd5-8d69fa01a9c1 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:18.828050 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45688ca-0571-411b-9a6d-81a5659b9ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:18.828713 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45688ca-0571-411b-9a6d-81a5659b9ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:18.828996 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45688ca-0571-411b-9a6d-81a5659b9ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:18.829271 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45688ca-0571-411b-9a6d-81a5659b9ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:18.829989 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45688ca-0571-411b-9a6d-81a5659b9ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:18.831613 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45688ca-0571-411b-9a6d-81a5659b9ab0 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:18.841354 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1412/2817] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:37:18.845659 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35bed8-f2c6-4368-9abf-dc83b30e5805 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:18.856854 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35bed8-f2c6-4368-9abf-dc83b30e5805 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:18.861477 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9d805-751c-4eb3-acd2-e3fab4845922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:18.862351 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9d805-751c-4eb3-acd2-e3fab4845922 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:18.862629 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9d805-751c-4eb3-acd2-e3fab4845922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:18.862904 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9d805-751c-4eb3-acd2-e3fab4845922 admin admin] PATH_INFO: `/v3/projects/25bd5415483a437ca3aa471e34adc661/users/e4e9fc456a7349fcbce9cdcfc616a9b9/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:18.864887 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9d805-751c-4eb3-acd2-e3fab4845922 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25bd5415483a437ca3aa471e34adc661, user_id=e4e9fc456a7349fcbce9cdcfc616a9b9, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:18.868353 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9d805-751c-4eb3-acd2-e3fab4845922 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:18.873709 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:18.879769 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1406/2818] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:37:18 2022] PUT /identity/v3/projects/25bd5415483a437ca3aa471e34adc661/users/e4e9fc456a7349fcbce9cdcfc616a9b9/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:18.890980 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066443ef-7524-4347-a20f-09f022d89670 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:18.891302 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066443ef-7524-4347-a20f-09f022d89670 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:18.891588 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066443ef-7524-4347-a20f-09f022d89670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:18.917959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-066443ef-7524-4347-a20f-09f022d89670 None None] MFA Rules not processed for user `e4e9fc456a7349fcbce9cdcfc616a9b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:18.919085 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-066443ef-7524-4347-a20f-09f022d89670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:18.966159 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:18.988810 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1413/2819] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:18 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:18.999888 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9d805-751c-4eb3-acd2-e3fab4845922 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:19.001598 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-80b9d805-751c-4eb3-acd2-e3fab4845922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:19.058656 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9d805-751c-4eb3-acd2-e3fab4845922 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:19.062528 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc2087-a54d-487a-ba78-68df5488924b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:19.063285 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc2087-a54d-487a-ba78-68df5488924b service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:19.063609 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc2087-a54d-487a-ba78-68df5488924b service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:19.063929 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc2087-a54d-487a-ba78-68df5488924b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:19.073823 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dc2087-a54d-487a-ba78-68df5488924b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:19.075303 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dc2087-a54d-487a-ba78-68df5488924b service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:19.084709 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1407/2820] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:37:18 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:37:19.154849 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066443ef-7524-4347-a20f-09f022d89670 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:19.165141 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066443ef-7524-4347-a20f-09f022d89670 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:19.168903 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda4ae7-aee5-456b-9549-bb236dda2297 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:19.169493 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda4ae7-aee5-456b-9549-bb236dda2297 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:19.169776 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda4ae7-aee5-456b-9549-bb236dda2297 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:19.170053 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda4ae7-aee5-456b-9549-bb236dda2297 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:19.225029 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbda4ae7-aee5-456b-9549-bb236dda2297 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:19.227484 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbda4ae7-aee5-456b-9549-bb236dda2297 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:19.236413 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1414/2821] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:37:19 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:37:21.935170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc2087-a54d-487a-ba78-68df5488924b service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:21.996033 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc2087-a54d-487a-ba78-68df5488924b service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:22.000559 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec26a7-2794-4b53-8c67-4b73aca983fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:22.001420 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec26a7-2794-4b53-8c67-4b73aca983fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:22.001703 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec26a7-2794-4b53-8c67-4b73aca983fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:22.001976 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec26a7-2794-4b53-8c67-4b73aca983fb admin admin] PATH_INFO: `/v3/users/59837924d4fe4e6aae6f3ac821fd9487` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:22.003153 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec26a7-2794-4b53-8c67-4b73aca983fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=59837924d4fe4e6aae6f3ac821fd9487)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:22.005123 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec26a7-2794-4b53-8c67-4b73aca983fb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:22.050735 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f6ec26a7-2794-4b53-8c67-4b73aca983fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59837924d4fe4e6aae6f3ac821fd9487'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:22.051413 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f6ec26a7-2794-4b53-8c67-4b73aca983fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59837924d4fe4e6aae6f3ac821fd9487'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:22.052025 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f6ec26a7-2794-4b53-8c67-4b73aca983fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59837924d4fe4e6aae6f3ac821fd9487'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:22.059074 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f6ec26a7-2794-4b53-8c67-4b73aca983fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59837924d4fe4e6aae6f3ac821fd9487'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:22.071649 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f6ec26a7-2794-4b53-8c67-4b73aca983fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59837924d4fe4e6aae6f3ac821fd9487'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:22.084091 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1408/2822] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:37:21 2022] DELETE /identity/v3/users/59837924d4fe4e6aae6f3ac821fd9487 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:22.089283 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816eee13-d22c-4097-a384-8d8ca6a3cc7a None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:22.089794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816eee13-d22c-4097-a384-8d8ca6a3cc7a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:22.089905 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816eee13-d22c-4097-a384-8d8ca6a3cc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:22.107373 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-816eee13-d22c-4097-a384-8d8ca6a3cc7a None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:22.108553 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-816eee13-d22c-4097-a384-8d8ca6a3cc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:22.166169 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1415/2823] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:22.175790 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec26a7-2794-4b53-8c67-4b73aca983fb admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:22.177373 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f6ec26a7-2794-4b53-8c67-4b73aca983fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:22.200372 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:22.237580 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec26a7-2794-4b53-8c67-4b73aca983fb admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:22.241142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c013fd9-6af8-494d-b069-54838e5e8b49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:22.242334 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c013fd9-6af8-494d-b069-54838e5e8b49 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:22.242613 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c013fd9-6af8-494d-b069-54838e5e8b49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:22.242886 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c013fd9-6af8-494d-b069-54838e5e8b49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:22.251713 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c013fd9-6af8-494d-b069-54838e5e8b49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:22.253129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c013fd9-6af8-494d-b069-54838e5e8b49 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:22.262902 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1409/2824] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:37:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:37:22.523831 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816eee13-d22c-4097-a384-8d8ca6a3cc7a None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:22.525544 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-816eee13-d22c-4097-a384-8d8ca6a3cc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:22.538152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816eee13-d22c-4097-a384-8d8ca6a3cc7a None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:22.544082 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041400b-edea-481d-9853-0ee21c1ee89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:22.544953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041400b-edea-481d-9853-0ee21c1ee89a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:22.545231 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041400b-edea-481d-9853-0ee21c1ee89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:22.545505 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041400b-edea-481d-9853-0ee21c1ee89a admin admin] PATH_INFO: `/v3/projects/7d2c0ee4ef5b40b7977ce411011938b7` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:22.546747 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6041400b-edea-481d-9853-0ee21c1ee89a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d2c0ee4ef5b40b7977ce411011938b7)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:22.548606 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6041400b-edea-481d-9853-0ee21c1ee89a admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:22.595907 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6041400b-edea-481d-9853-0ee21c1ee89a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d2c0ee4ef5b40b7977ce411011938b7', 'request_id': 'req-6041400b-edea-481d-9853-0ee21c1ee89a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:22.603965 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6041400b-edea-481d-9853-0ee21c1ee89a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d2c0ee4ef5b40b7977ce411011938b7', 'request_id': 'req-6041400b-edea-481d-9853-0ee21c1ee89a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:22.610033 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6041400b-edea-481d-9853-0ee21c1ee89a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d2c0ee4ef5b40b7977ce411011938b7', 'request_id': 'req-6041400b-edea-481d-9853-0ee21c1ee89a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:22.620852 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6041400b-edea-481d-9853-0ee21c1ee89a admin admin] The token cache is being invalidate because project 7d2c0ee4ef5b40b7977ce411011938b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:37:22.621135 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6041400b-edea-481d-9853-0ee21c1ee89a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:22.621762 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6041400b-edea-481d-9853-0ee21c1ee89a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:22.623311 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1416/2825] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:37:22 2022] DELETE /identity/v3/projects/7d2c0ee4ef5b40b7977ce411011938b7 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:22.627278 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c013fd9-6af8-494d-b069-54838e5e8b49 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:22.637133 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c013fd9-6af8-494d-b069-54838e5e8b49 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:22.692400 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a460e-bf13-45c1-a778-d614ad8bce3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:22.693271 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a460e-bf13-45c1-a778-d614ad8bce3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:22.693557 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a460e-bf13-45c1-a778-d614ad8bce3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:22.693832 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a460e-bf13-45c1-a778-d614ad8bce3b admin admin] PATH_INFO: `/v3/users/e4e9fc456a7349fcbce9cdcfc616a9b9` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:22.695549 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060a460e-bf13-45c1-a778-d614ad8bce3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4e9fc456a7349fcbce9cdcfc616a9b9)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:22.697561 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060a460e-bf13-45c1-a778-d614ad8bce3b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:22.742470 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-060a460e-bf13-45c1-a778-d614ad8bce3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4e9fc456a7349fcbce9cdcfc616a9b9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:22.743274 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-060a460e-bf13-45c1-a778-d614ad8bce3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4e9fc456a7349fcbce9cdcfc616a9b9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:22.743966 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-060a460e-bf13-45c1-a778-d614ad8bce3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4e9fc456a7349fcbce9cdcfc616a9b9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:22.750705 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-060a460e-bf13-45c1-a778-d614ad8bce3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4e9fc456a7349fcbce9cdcfc616a9b9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:22.762875 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-060a460e-bf13-45c1-a778-d614ad8bce3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4e9fc456a7349fcbce9cdcfc616a9b9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:22.775535 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1410/2826] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:37:22 2022] DELETE /identity/v3/users/e4e9fc456a7349fcbce9cdcfc616a9b9 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:23.051745 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041400b-edea-481d-9853-0ee21c1ee89a admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:23.053240 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6041400b-edea-481d-9853-0ee21c1ee89a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:23.075821 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041400b-edea-481d-9853-0ee21c1ee89a admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:23.082297 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:23.138617 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4f8d7-053c-4041-b446-c7ba1a3960c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:23.139430 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4f8d7-053c-4041-b446-c7ba1a3960c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:23.139709 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4f8d7-053c-4041-b446-c7ba1a3960c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:23.139983 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4f8d7-053c-4041-b446-c7ba1a3960c9 admin admin] PATH_INFO: `/v3/projects/25bd5415483a437ca3aa471e34adc661` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:23.141177 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4f8d7-053c-4041-b446-c7ba1a3960c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25bd5415483a437ca3aa471e34adc661)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:23.143088 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4f8d7-053c-4041-b446-c7ba1a3960c9 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:23.189381 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d8b4f8d7-053c-4041-b446-c7ba1a3960c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25bd5415483a437ca3aa471e34adc661', 'request_id': 'req-d8b4f8d7-053c-4041-b446-c7ba1a3960c9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:23.197149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d8b4f8d7-053c-4041-b446-c7ba1a3960c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25bd5415483a437ca3aa471e34adc661', 'request_id': 'req-d8b4f8d7-053c-4041-b446-c7ba1a3960c9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:23.203608 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d8b4f8d7-053c-4041-b446-c7ba1a3960c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25bd5415483a437ca3aa471e34adc661', 'request_id': 'req-d8b4f8d7-053c-4041-b446-c7ba1a3960c9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:23.214303 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d8b4f8d7-053c-4041-b446-c7ba1a3960c9 admin admin] The token cache is being invalidate because project 25bd5415483a437ca3aa471e34adc661 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:37:23.214589 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d8b4f8d7-053c-4041-b446-c7ba1a3960c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:23.215233 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d8b4f8d7-053c-4041-b446-c7ba1a3960c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:23.216778 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1417/2827] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:37:23 2022] DELETE /identity/v3/projects/25bd5415483a437ca3aa471e34adc661 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:23.221741 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd41864-d43a-4d14-a134-3872e8a82c22 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:23.222211 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd41864-d43a-4d14-a134-3872e8a82c22 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:23.222316 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd41864-d43a-4d14-a134-3872e8a82c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:23.239663 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-fdd41864-d43a-4d14-a134-3872e8a82c22 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:23.240928 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fdd41864-d43a-4d14-a134-3872e8a82c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:23.279755 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:23.302549 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1411/2828] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:23.310492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ffcd2-65cd-4e3a-b54a-282bda9df5cc None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:23.310806 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ffcd2-65cd-4e3a-b54a-282bda9df5cc None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:23.311089 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ffcd2-65cd-4e3a-b54a-282bda9df5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:23.329865 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-677ffcd2-65cd-4e3a-b54a-282bda9df5cc None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:23.331728 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-677ffcd2-65cd-4e3a-b54a-282bda9df5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:23.340450 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1418/2829] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:37:23.344596 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd41864-d43a-4d14-a134-3872e8a82c22 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:23.354593 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd41864-d43a-4d14-a134-3872e8a82c22 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:23.359167 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e863cc8-8289-4948-83fd-00aca74d23ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:23.359848 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e863cc8-8289-4948-83fd-00aca74d23ec admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:23.360124 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e863cc8-8289-4948-83fd-00aca74d23ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:23.360394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e863cc8-8289-4948-83fd-00aca74d23ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:23.361108 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e863cc8-8289-4948-83fd-00aca74d23ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:37:23.361406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e863cc8-8289-4948-83fd-00aca74d23ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:23.363336 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e863cc8-8289-4948-83fd-00aca74d23ec admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:23.374939 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1412/2830] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:37:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:37:23.379024 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ffcd2-65cd-4e3a-b54a-282bda9df5cc None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:23.388809 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ffcd2-65cd-4e3a-b54a-282bda9df5cc None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:23.393317 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f874a2-2697-45f6-8a45-0096d2558d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:23.393959 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f874a2-2697-45f6-8a45-0096d2558d8e admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:23.394238 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f874a2-2697-45f6-8a45-0096d2558d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:23.394510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f874a2-2697-45f6-8a45-0096d2558d8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:23.395186 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f874a2-2697-45f6-8a45-0096d2558d8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:23.397109 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f874a2-2697-45f6-8a45-0096d2558d8e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:23.423984 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1419/2831] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:37:23 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:37:23.428087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e863cc8-8289-4948-83fd-00aca74d23ec admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:23.438032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e863cc8-8289-4948-83fd-00aca74d23ec admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:23.494432 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aded270c-ea3f-4b14-82af-493068882991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:23.495100 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded270c-ea3f-4b14-82af-493068882991 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:23.495416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded270c-ea3f-4b14-82af-493068882991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:23.495709 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded270c-ea3f-4b14-82af-493068882991 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:23.496395 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aded270c-ea3f-4b14-82af-493068882991 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:23.498294 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aded270c-ea3f-4b14-82af-493068882991 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:23.518855 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1413/2832] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:37:23 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:37:23.523482 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f874a2-2697-45f6-8a45-0096d2558d8e admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:23.533793 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f874a2-2697-45f6-8a45-0096d2558d8e admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:23.538676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91363d-221f-4a78-a1f3-04ad4f3dbf91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:23.539602 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91363d-221f-4a78-a1f3-04ad4f3dbf91 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:23.539886 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91363d-221f-4a78-a1f3-04ad4f3dbf91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:23.540158 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91363d-221f-4a78-a1f3-04ad4f3dbf91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:23.540875 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91363d-221f-4a78-a1f3-04ad4f3dbf91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:23.542451 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91363d-221f-4a78-a1f3-04ad4f3dbf91 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:23.552714 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1420/2833] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:37:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:37:23.557165 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aded270c-ea3f-4b14-82af-493068882991 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:23.567027 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aded270c-ea3f-4b14-82af-493068882991 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:23.572273 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efaa9ef-ec97-4b8d-a249-6efdeef4535a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:23.572928 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efaa9ef-ec97-4b8d-a249-6efdeef4535a admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:23.573215 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efaa9ef-ec97-4b8d-a249-6efdeef4535a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:23.573492 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efaa9ef-ec97-4b8d-a249-6efdeef4535a admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:23.574217 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efaa9ef-ec97-4b8d-a249-6efdeef4535a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:23.575843 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efaa9ef-ec97-4b8d-a249-6efdeef4535a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:23.585897 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1414/2834] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:37:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:37:23.589892 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91363d-221f-4a78-a1f3-04ad4f3dbf91 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:23.599697 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91363d-221f-4a78-a1f3-04ad4f3dbf91 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:23.604752 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0020e3f4-a32c-45df-847c-19ea032f8cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:23.605615 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020e3f4-a32c-45df-847c-19ea032f8cf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:23.605889 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020e3f4-a32c-45df-847c-19ea032f8cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:23.606158 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020e3f4-a32c-45df-847c-19ea032f8cf8 admin admin] PATH_INFO: `/v3/projects/a4df6b35602d45e7a0640c278fd3ffc5/users/49dac8485af5413dae8e345024c89d15/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:23.622050 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020e3f4-a32c-45df-847c-19ea032f8cf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4df6b35602d45e7a0640c278fd3ffc5, user_id=49dac8485af5413dae8e345024c89d15, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:23.624693 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020e3f4-a32c-45df-847c-19ea032f8cf8 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:23.637633 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1421/2835] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:37:23 2022] PUT /identity/v3/projects/a4df6b35602d45e7a0640c278fd3ffc5/users/49dac8485af5413dae8e345024c89d15/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:23.648430 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0a96d-528b-4e5f-9f91-3cca7ec3721a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:23.648760 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0a96d-528b-4e5f-9f91-3cca7ec3721a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:23.649050 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0a96d-528b-4e5f-9f91-3cca7ec3721a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:23.675248 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-dbd0a96d-528b-4e5f-9f91-3cca7ec3721a None None] MFA Rules not processed for user `49dac8485af5413dae8e345024c89d15`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:23.676484 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dbd0a96d-528b-4e5f-9f91-3cca7ec3721a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:23.743032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1415/2836] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:23.747526 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0020e3f4-a32c-45df-847c-19ea032f8cf8 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:23.782050 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:23.811940 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0020e3f4-a32c-45df-847c-19ea032f8cf8 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:23.816585 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31adb2-4f66-459f-8c8f-71bec61856a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:23.817244 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31adb2-4f66-459f-8c8f-71bec61856a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:23.817527 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31adb2-4f66-459f-8c8f-71bec61856a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:23.817800 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31adb2-4f66-459f-8c8f-71bec61856a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:23.818497 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31adb2-4f66-459f-8c8f-71bec61856a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:23.820460 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31adb2-4f66-459f-8c8f-71bec61856a2 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:23.845681 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1422/2837] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:37:23 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:37:23.849900 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0a96d-528b-4e5f-9f91-3cca7ec3721a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:23.867200 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0a96d-528b-4e5f-9f91-3cca7ec3721a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:23.900249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:23.923755 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa9b07-5f11-420a-b119-e816b4cd5f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:23.924421 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa9b07-5f11-420a-b119-e816b4cd5f1b admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:23.924699 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa9b07-5f11-420a-b119-e816b4cd5f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:23.924972 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa9b07-5f11-420a-b119-e816b4cd5f1b admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:23.925650 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa9b07-5f11-420a-b119-e816b4cd5f1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:23.927578 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa9b07-5f11-420a-b119-e816b4cd5f1b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:23.950898 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1416/2838] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:37:23 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:37:23.952137 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31adb2-4f66-459f-8c8f-71bec61856a2 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:23.963292 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31adb2-4f66-459f-8c8f-71bec61856a2 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:23.968084 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e6b00-d064-47b2-a0ac-22779e6f0545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:23.968772 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e6b00-d064-47b2-a0ac-22779e6f0545 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:23.969057 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e6b00-d064-47b2-a0ac-22779e6f0545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:23.969340 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e6b00-d064-47b2-a0ac-22779e6f0545 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:23.970072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e6b00-d064-47b2-a0ac-22779e6f0545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:23.971704 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e6b00-d064-47b2-a0ac-22779e6f0545 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:23.983367 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1423/2839] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:37:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:37:23.987265 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa9b07-5f11-420a-b119-e816b4cd5f1b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:23.997299 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa9b07-5f11-420a-b119-e816b4cd5f1b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:24.001865 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d5f1b-ce17-41fd-8828-a19bd5ddfbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:24.002741 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d5f1b-ce17-41fd-8828-a19bd5ddfbc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:24.003025 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d5f1b-ce17-41fd-8828-a19bd5ddfbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:24.003358 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d5f1b-ce17-41fd-8828-a19bd5ddfbc8 admin admin] PATH_INFO: `/v3/projects/5158aae44e1c4939ae589e86df3ddbdd/users/2d055c75cc214373874917b2e8359f26/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:24.018606 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d5f1b-ce17-41fd-8828-a19bd5ddfbc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5158aae44e1c4939ae589e86df3ddbdd, user_id=2d055c75cc214373874917b2e8359f26, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:24.021325 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d5f1b-ce17-41fd-8828-a19bd5ddfbc8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:24.034410 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1417/2840] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:37:23 2022] PUT /identity/v3/projects/5158aae44e1c4939ae589e86df3ddbdd/users/2d055c75cc214373874917b2e8359f26/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:24.038658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e6b00-d064-47b2-a0ac-22779e6f0545 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:24.048794 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e6b00-d064-47b2-a0ac-22779e6f0545 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:24.108309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79dd20-db9b-4c3c-a397-5c59eab62e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:24.108983 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79dd20-db9b-4c3c-a397-5c59eab62e53 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:24.109268 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79dd20-db9b-4c3c-a397-5c59eab62e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:24.109680 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79dd20-db9b-4c3c-a397-5c59eab62e53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:24.110408 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c79dd20-db9b-4c3c-a397-5c59eab62e53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:24.112065 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c79dd20-db9b-4c3c-a397-5c59eab62e53 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:24.121903 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1424/2841] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:37:24.125999 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d5f1b-ce17-41fd-8828-a19bd5ddfbc8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:24.136630 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d5f1b-ce17-41fd-8828-a19bd5ddfbc8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:24.141152 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d7fbf-9c6d-4d09-9b4c-7517fd6bf39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:24.142081 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d7fbf-9c6d-4d09-9b4c-7517fd6bf39e admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:24.142416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d7fbf-9c6d-4d09-9b4c-7517fd6bf39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:24.142613 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d7fbf-9c6d-4d09-9b4c-7517fd6bf39e admin admin] PATH_INFO: `/v3/projects/5158aae44e1c4939ae589e86df3ddbdd/users/2d055c75cc214373874917b2e8359f26/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:24.144649 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d7fbf-9c6d-4d09-9b4c-7517fd6bf39e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5158aae44e1c4939ae589e86df3ddbdd, user_id=2d055c75cc214373874917b2e8359f26, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:24.147270 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d7fbf-9c6d-4d09-9b4c-7517fd6bf39e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:24.158422 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1418/2842] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:37:24 2022] PUT /identity/v3/projects/5158aae44e1c4939ae589e86df3ddbdd/users/2d055c75cc214373874917b2e8359f26/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:24.169789 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cebb6c-1b5f-4dfb-88e3-acaad6075f2b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:24.170208 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cebb6c-1b5f-4dfb-88e3-acaad6075f2b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:24.170379 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cebb6c-1b5f-4dfb-88e3-acaad6075f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:24.197410 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-98cebb6c-1b5f-4dfb-88e3-acaad6075f2b None None] MFA Rules not processed for user `2d055c75cc214373874917b2e8359f26`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:24.198609 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-98cebb6c-1b5f-4dfb-88e3-acaad6075f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:24.266492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1425/2843] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:24 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:24.278899 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d7fbf-9c6d-4d09-9b4c-7517fd6bf39e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:24.280626 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c12d7fbf-9c6d-4d09-9b4c-7517fd6bf39e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:24.341358 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d7fbf-9c6d-4d09-9b4c-7517fd6bf39e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:24.345257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762aa03-31d3-4d93-940b-448625d641b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:24.345902 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762aa03-31d3-4d93-940b-448625d641b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:24.346184 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762aa03-31d3-4d93-940b-448625d641b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:24.346496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762aa03-31d3-4d93-940b-448625d641b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:24.356864 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1762aa03-31d3-4d93-940b-448625d641b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:24.358281 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1762aa03-31d3-4d93-940b-448625d641b9 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:24.368565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1419/2844] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:37:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:37:24.665621 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cebb6c-1b5f-4dfb-88e3-acaad6075f2b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:24.674645 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cebb6c-1b5f-4dfb-88e3-acaad6075f2b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:24.680130 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51911299-9072-46c5-bdb7-8172bf4b891c tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d055c75cc214373874917b2e8359f26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5158aae44e1c4939ae589e86df3ddbdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:24.680710 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51911299-9072-46c5-bdb7-8172bf4b891c tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:24.681000 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51911299-9072-46c5-bdb7-8172bf4b891c tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:24.681281 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51911299-9072-46c5-bdb7-8172bf4b891c tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:24.682047 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51911299-9072-46c5-bdb7-8172bf4b891c tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:37:24.682352 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51911299-9072-46c5-bdb7-8172bf4b891c tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:24.683989 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51911299-9072-46c5-bdb7-8172bf4b891c tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:24.695599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1426/2845] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:37:24.699480 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762aa03-31d3-4d93-940b-448625d641b9 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:24.708512 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762aa03-31d3-4d93-940b-448625d641b9 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:24.713147 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d3b4b-d6a1-42ef-a444-502a2c214cdd tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d055c75cc214373874917b2e8359f26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5158aae44e1c4939ae589e86df3ddbdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:24.713795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d3b4b-d6a1-42ef-a444-502a2c214cdd tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:24.714084 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d3b4b-d6a1-42ef-a444-502a2c214cdd tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:24.714394 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d3b4b-d6a1-42ef-a444-502a2c214cdd tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:24.715093 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d3b4b-d6a1-42ef-a444-502a2c214cdd tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:24.717014 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d3b4b-d6a1-42ef-a444-502a2c214cdd tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:24.743765 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1420/2846] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:37:24 2022] POST /identity/v3/projects => generated 354 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:37:24.747969 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51911299-9072-46c5-bdb7-8172bf4b891c tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:24.758174 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51911299-9072-46c5-bdb7-8172bf4b891c tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:24.769582 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:24.815822 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985dfeb-603c-449a-b22d-b68dccb7a9db tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d055c75cc214373874917b2e8359f26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5158aae44e1c4939ae589e86df3ddbdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:24.816481 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985dfeb-603c-449a-b22d-b68dccb7a9db tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:24.816770 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985dfeb-603c-449a-b22d-b68dccb7a9db tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:24.817050 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985dfeb-603c-449a-b22d-b68dccb7a9db tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:24.817773 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985dfeb-603c-449a-b22d-b68dccb7a9db tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:37:24.818082 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985dfeb-603c-449a-b22d-b68dccb7a9db tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:24.819703 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985dfeb-603c-449a-b22d-b68dccb7a9db tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:24.831714 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1427/2847] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:37:24.867994 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d3b4b-d6a1-42ef-a444-502a2c214cdd tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:24.878008 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d3b4b-d6a1-42ef-a444-502a2c214cdd tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:24.883234 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306cc26d-07e0-4aca-85fb-a312e0eb9f5f tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d055c75cc214373874917b2e8359f26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5158aae44e1c4939ae589e86df3ddbdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:24.884249 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cc26d-07e0-4aca-85fb-a312e0eb9f5f tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:24.884553 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cc26d-07e0-4aca-85fb-a312e0eb9f5f tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:24.884825 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cc26d-07e0-4aca-85fb-a312e0eb9f5f tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] PATH_INFO: `/v3/projects/aeb1b1e331a64648b8482e4d38525110` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:24.886276 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306cc26d-07e0-4aca-85fb-a312e0eb9f5f tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] RBAC: Authorizing `identity:delete_project(project_id=aeb1b1e331a64648b8482e4d38525110)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:24.888505 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306cc26d-07e0-4aca-85fb-a312e0eb9f5f tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:24.935709 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-306cc26d-07e0-4aca-85fb-a312e0eb9f5f tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeb1b1e331a64648b8482e4d38525110', 'request_id': 'req-306cc26d-07e0-4aca-85fb-a312e0eb9f5f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:24.942795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-306cc26d-07e0-4aca-85fb-a312e0eb9f5f tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeb1b1e331a64648b8482e4d38525110', 'request_id': 'req-306cc26d-07e0-4aca-85fb-a312e0eb9f5f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:24.948973 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-306cc26d-07e0-4aca-85fb-a312e0eb9f5f tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeb1b1e331a64648b8482e4d38525110', 'request_id': 'req-306cc26d-07e0-4aca-85fb-a312e0eb9f5f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:24.950080 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:24.963084 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-306cc26d-07e0-4aca-85fb-a312e0eb9f5f tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] The token cache is being invalidate because project aeb1b1e331a64648b8482e4d38525110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:37:24.963400 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-306cc26d-07e0-4aca-85fb-a312e0eb9f5f tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:24.964060 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-306cc26d-07e0-4aca-85fb-a312e0eb9f5f tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:24.965645 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1421/2848] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:37:24 2022] DELETE /identity/v3/projects/aeb1b1e331a64648b8482e4d38525110 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:25.158624 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985dfeb-603c-449a-b22d-b68dccb7a9db tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:25.160400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2985dfeb-603c-449a-b22d-b68dccb7a9db tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:25.225816 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985dfeb-603c-449a-b22d-b68dccb7a9db tempest-VolumeQuotaClassesTest-1274438683 tempest-VolumeQuotaClassesTest-1274438683-project-admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:25.282092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b2ab6-bf73-40a6-8d51-28c79faba89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:25.282963 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b2ab6-bf73-40a6-8d51-28c79faba89a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:25.283343 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b2ab6-bf73-40a6-8d51-28c79faba89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:25.283652 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b2ab6-bf73-40a6-8d51-28c79faba89a admin admin] PATH_INFO: `/v3/users/49dac8485af5413dae8e345024c89d15` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:25.284837 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b2ab6-bf73-40a6-8d51-28c79faba89a admin admin] RBAC: Authorizing `identity:delete_user(user_id=49dac8485af5413dae8e345024c89d15)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:25.286742 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b2ab6-bf73-40a6-8d51-28c79faba89a admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:25.332942 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-213b2ab6-bf73-40a6-8d51-28c79faba89a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49dac8485af5413dae8e345024c89d15'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:25.341092 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:25.346091 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-213b2ab6-bf73-40a6-8d51-28c79faba89a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49dac8485af5413dae8e345024c89d15'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:25.346824 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-213b2ab6-bf73-40a6-8d51-28c79faba89a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49dac8485af5413dae8e345024c89d15'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:25.347935 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-213b2ab6-bf73-40a6-8d51-28c79faba89a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49dac8485af5413dae8e345024c89d15'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:25.355607 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-213b2ab6-bf73-40a6-8d51-28c79faba89a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49dac8485af5413dae8e345024c89d15'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:25.370264 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1428/2849] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:37:25 2022] DELETE /identity/v3/users/49dac8485af5413dae8e345024c89d15 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:25.374755 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f9adf-ded7-415c-beaf-248957dbd0f9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:25.375349 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f9adf-ded7-415c-beaf-248957dbd0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:25.375462 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f9adf-ded7-415c-beaf-248957dbd0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:25.393784 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4f9f9adf-ded7-415c-beaf-248957dbd0f9 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:25.394985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4f9f9adf-ded7-415c-beaf-248957dbd0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:25.454298 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1422/2850] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:25.463922 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b2ab6-bf73-40a6-8d51-28c79faba89a admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:25.465436 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-213b2ab6-bf73-40a6-8d51-28c79faba89a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:25.522728 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b2ab6-bf73-40a6-8d51-28c79faba89a admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:25.526635 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9890a-01c9-407a-bda7-2734683b2931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:25.527308 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9890a-01c9-407a-bda7-2734683b2931 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:25.527601 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9890a-01c9-407a-bda7-2734683b2931 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:25.527881 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9890a-01c9-407a-bda7-2734683b2931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:25.536895 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d9890a-01c9-407a-bda7-2734683b2931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:25.538363 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d9890a-01c9-407a-bda7-2734683b2931 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:25.549540 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1429/2851] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:37:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:37:25.814278 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f9adf-ded7-415c-beaf-248957dbd0f9 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:25.815742 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4f9f9adf-ded7-415c-beaf-248957dbd0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:25.827845 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f9adf-ded7-415c-beaf-248957dbd0f9 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:25.832796 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74501f-56c4-451e-98fc-46e46f8df368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:25.833769 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74501f-56c4-451e-98fc-46e46f8df368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:25.834119 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74501f-56c4-451e-98fc-46e46f8df368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:25.834497 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74501f-56c4-451e-98fc-46e46f8df368 admin admin] PATH_INFO: `/v3/projects/a4df6b35602d45e7a0640c278fd3ffc5` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:25.836543 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74501f-56c4-451e-98fc-46e46f8df368 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4df6b35602d45e7a0640c278fd3ffc5)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:25.838820 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74501f-56c4-451e-98fc-46e46f8df368 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:25.885690 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fd74501f-56c4-451e-98fc-46e46f8df368 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4df6b35602d45e7a0640c278fd3ffc5', 'request_id': 'req-fd74501f-56c4-451e-98fc-46e46f8df368'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:25.892763 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fd74501f-56c4-451e-98fc-46e46f8df368 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4df6b35602d45e7a0640c278fd3ffc5', 'request_id': 'req-fd74501f-56c4-451e-98fc-46e46f8df368'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:25.898771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fd74501f-56c4-451e-98fc-46e46f8df368 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4df6b35602d45e7a0640c278fd3ffc5', 'request_id': 'req-fd74501f-56c4-451e-98fc-46e46f8df368'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:25.909435 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fd74501f-56c4-451e-98fc-46e46f8df368 admin admin] The token cache is being invalidate because project a4df6b35602d45e7a0640c278fd3ffc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:37:25.909723 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fd74501f-56c4-451e-98fc-46e46f8df368 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:25.910351 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fd74501f-56c4-451e-98fc-46e46f8df368 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:25.911952 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1423/2852] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:37:25 2022] DELETE /identity/v3/projects/a4df6b35602d45e7a0640c278fd3ffc5 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:25.916639 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9890a-01c9-407a-bda7-2734683b2931 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:25.926496 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9890a-01c9-407a-bda7-2734683b2931 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:25.984145 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883cce51-99bd-49a8-8bf1-e26ade9f134e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:25.985015 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cce51-99bd-49a8-8bf1-e26ade9f134e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:25.985301 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cce51-99bd-49a8-8bf1-e26ade9f134e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:25.985578 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cce51-99bd-49a8-8bf1-e26ade9f134e admin admin] PATH_INFO: `/v3/users/2d055c75cc214373874917b2e8359f26` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:25.986761 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883cce51-99bd-49a8-8bf1-e26ade9f134e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d055c75cc214373874917b2e8359f26)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:25.988716 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883cce51-99bd-49a8-8bf1-e26ade9f134e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:26.034211 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-883cce51-99bd-49a8-8bf1-e26ade9f134e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d055c75cc214373874917b2e8359f26'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:26.046504 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-883cce51-99bd-49a8-8bf1-e26ade9f134e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d055c75cc214373874917b2e8359f26'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:26.047185 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-883cce51-99bd-49a8-8bf1-e26ade9f134e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d055c75cc214373874917b2e8359f26'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:26.047815 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-883cce51-99bd-49a8-8bf1-e26ade9f134e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d055c75cc214373874917b2e8359f26'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:26.054641 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-883cce51-99bd-49a8-8bf1-e26ade9f134e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d055c75cc214373874917b2e8359f26'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:26.062510 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:26.069144 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1430/2853] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:37:25 2022] DELETE /identity/v3/users/2d055c75cc214373874917b2e8359f26 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:26.334662 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74501f-56c4-451e-98fc-46e46f8df368 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:26.336624 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fd74501f-56c4-451e-98fc-46e46f8df368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:26.352633 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74501f-56c4-451e-98fc-46e46f8df368 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:26.409573 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034054f4-ed81-4316-a166-8432b4c6d87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:26.410431 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034054f4-ed81-4316-a166-8432b4c6d87c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:26.410714 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034054f4-ed81-4316-a166-8432b4c6d87c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:26.410986 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034054f4-ed81-4316-a166-8432b4c6d87c admin admin] PATH_INFO: `/v3/projects/5158aae44e1c4939ae589e86df3ddbdd` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:26.412185 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034054f4-ed81-4316-a166-8432b4c6d87c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5158aae44e1c4939ae589e86df3ddbdd)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:26.414661 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034054f4-ed81-4316-a166-8432b4c6d87c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:26.435446 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:26.463964 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-034054f4-ed81-4316-a166-8432b4c6d87c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5158aae44e1c4939ae589e86df3ddbdd', 'request_id': 'req-034054f4-ed81-4316-a166-8432b4c6d87c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:26.471329 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-034054f4-ed81-4316-a166-8432b4c6d87c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5158aae44e1c4939ae589e86df3ddbdd', 'request_id': 'req-034054f4-ed81-4316-a166-8432b4c6d87c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:26.477691 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-034054f4-ed81-4316-a166-8432b4c6d87c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5158aae44e1c4939ae589e86df3ddbdd', 'request_id': 'req-034054f4-ed81-4316-a166-8432b4c6d87c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:26.488569 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-034054f4-ed81-4316-a166-8432b4c6d87c admin admin] The token cache is being invalidate because project 5158aae44e1c4939ae589e86df3ddbdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:37:26.488853 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-034054f4-ed81-4316-a166-8432b4c6d87c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:26.489507 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-034054f4-ed81-4316-a166-8432b4c6d87c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:26.491257 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1424/2854] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:37:26 2022] DELETE /identity/v3/projects/5158aae44e1c4939ae589e86df3ddbdd => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:26.496188 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a85122-5ec9-432b-8f55-8bfb0064659a None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:26.496661 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a85122-5ec9-432b-8f55-8bfb0064659a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:26.496796 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a85122-5ec9-432b-8f55-8bfb0064659a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:26.514868 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-e3a85122-5ec9-432b-8f55-8bfb0064659a None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:26.516108 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e3a85122-5ec9-432b-8f55-8bfb0064659a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:26.575108 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1431/2855] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:26.582905 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c4df9-0b1b-47f4-9ad0-5f4a57079b8d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:26.583142 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c4df9-0b1b-47f4-9ad0-5f4a57079b8d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:26.583454 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c4df9-0b1b-47f4-9ad0-5f4a57079b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:26.602711 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-524c4df9-0b1b-47f4-9ad0-5f4a57079b8d None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:26.603937 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-524c4df9-0b1b-47f4-9ad0-5f4a57079b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:26.611830 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1425/2856] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:26.615833 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a85122-5ec9-432b-8f55-8bfb0064659a None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:26.626302 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a85122-5ec9-432b-8f55-8bfb0064659a None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:26.631029 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42fd06-c443-457e-bfb6-3b3380541176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:26.631724 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42fd06-c443-457e-bfb6-3b3380541176 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:26.632006 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42fd06-c443-457e-bfb6-3b3380541176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:26.632307 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42fd06-c443-457e-bfb6-3b3380541176 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:26.633583 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42fd06-c443-457e-bfb6-3b3380541176 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:37:26.633882 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42fd06-c443-457e-bfb6-3b3380541176 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:26.635520 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42fd06-c443-457e-bfb6-3b3380541176 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:26.647056 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1432/2857] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:37:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:37:26.651989 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c4df9-0b1b-47f4-9ad0-5f4a57079b8d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:26.661064 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c4df9-0b1b-47f4-9ad0-5f4a57079b8d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:26.665905 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c95fac-0c01-4db4-8168-184f2949acb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:26.666544 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c95fac-0c01-4db4-8168-184f2949acb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:26.666823 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c95fac-0c01-4db4-8168-184f2949acb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:26.667095 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c95fac-0c01-4db4-8168-184f2949acb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:26.667819 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c95fac-0c01-4db4-8168-184f2949acb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:26.669700 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c95fac-0c01-4db4-8168-184f2949acb2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:26.696406 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1426/2858] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:37:26 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:37:26.700599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42fd06-c443-457e-bfb6-3b3380541176 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:26.710492 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42fd06-c443-457e-bfb6-3b3380541176 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:26.765793 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115d346-8f13-433b-bd7f-786ac2638eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:26.766451 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115d346-8f13-433b-bd7f-786ac2638eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:26.766731 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115d346-8f13-433b-bd7f-786ac2638eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:26.767004 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115d346-8f13-433b-bd7f-786ac2638eb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:26.767716 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115d346-8f13-433b-bd7f-786ac2638eb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:26.769649 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115d346-8f13-433b-bd7f-786ac2638eb8 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:26.788752 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1433/2859] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:37:26 2022] POST /identity/v3/users => generated 294 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:37:26.792868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c95fac-0c01-4db4-8168-184f2949acb2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:26.802868 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c95fac-0c01-4db4-8168-184f2949acb2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:26.808368 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3d8af-0f57-49c4-aa16-102878b57af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:26.809051 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3d8af-0f57-49c4-aa16-102878b57af0 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:26.809331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3d8af-0f57-49c4-aa16-102878b57af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:26.809731 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3d8af-0f57-49c4-aa16-102878b57af0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:26.810331 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3d8af-0f57-49c4-aa16-102878b57af0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:26.811959 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3d8af-0f57-49c4-aa16-102878b57af0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:26.823087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1427/2860] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:37:26.826985 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115d346-8f13-433b-bd7f-786ac2638eb8 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:26.837132 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115d346-8f13-433b-bd7f-786ac2638eb8 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:26.841790 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62056e-7d92-4014-957d-4b97461b9e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:26.842448 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62056e-7d92-4014-957d-4b97461b9e13 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:26.842761 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62056e-7d92-4014-957d-4b97461b9e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:26.843039 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62056e-7d92-4014-957d-4b97461b9e13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:26.843799 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef62056e-7d92-4014-957d-4b97461b9e13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:26.845382 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef62056e-7d92-4014-957d-4b97461b9e13 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:26.856045 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1434/2861] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:37:26.860181 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3d8af-0f57-49c4-aa16-102878b57af0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:26.870783 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3d8af-0f57-49c4-aa16-102878b57af0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:26.875721 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1efd886-f8e3-4881-8320-365b8fbc6898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:26.876603 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1efd886-f8e3-4881-8320-365b8fbc6898 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:26.876890 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1efd886-f8e3-4881-8320-365b8fbc6898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:26.877170 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1efd886-f8e3-4881-8320-365b8fbc6898 admin admin] PATH_INFO: `/v3/projects/3ce8738e3b814eedacd33571ec02ff25/users/e9593be5392b45f58e416d4b689955fa/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:26.891678 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1efd886-f8e3-4881-8320-365b8fbc6898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ce8738e3b814eedacd33571ec02ff25, user_id=e9593be5392b45f58e416d4b689955fa, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:26.894393 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1efd886-f8e3-4881-8320-365b8fbc6898 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:26.906818 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1428/2862] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:37:26 2022] PUT /identity/v3/projects/3ce8738e3b814eedacd33571ec02ff25/users/e9593be5392b45f58e416d4b689955fa/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:26.917855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca11ae-3327-4344-b6cb-c11244c66daf None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:26.918148 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca11ae-3327-4344-b6cb-c11244c66daf None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:26.918431 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca11ae-3327-4344-b6cb-c11244c66daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:26.945020 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-edca11ae-3327-4344-b6cb-c11244c66daf None None] MFA Rules not processed for user `e9593be5392b45f58e416d4b689955fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:26.946627 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-edca11ae-3327-4344-b6cb-c11244c66daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:27.002855 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:27.014744 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1435/2863] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:27.019202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1efd886-f8e3-4881-8320-365b8fbc6898 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:27.083618 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1efd886-f8e3-4881-8320-365b8fbc6898 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:27.088560 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c588d34a-e96d-4c50-a085-b234c15ca835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:27.089158 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588d34a-e96d-4c50-a085-b234c15ca835 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:27.089439 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588d34a-e96d-4c50-a085-b234c15ca835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:27.089716 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588d34a-e96d-4c50-a085-b234c15ca835 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:27.090399 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c588d34a-e96d-4c50-a085-b234c15ca835 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:27.092427 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c588d34a-e96d-4c50-a085-b234c15ca835 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:27.119115 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1429/2864] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:37:27 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:37:27.123090 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca11ae-3327-4344-b6cb-c11244c66daf None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:27.133629 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca11ae-3327-4344-b6cb-c11244c66daf None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:27.192550 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c10ec-f04b-4a37-a319-da5849a66e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:27.193220 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c10ec-f04b-4a37-a319-da5849a66e5e admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:27.193506 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c10ec-f04b-4a37-a319-da5849a66e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:27.193783 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c10ec-f04b-4a37-a319-da5849a66e5e admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:27.194476 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c10ec-f04b-4a37-a319-da5849a66e5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:27.196616 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c10ec-f04b-4a37-a319-da5849a66e5e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:27.216573 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1436/2865] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:37:27 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:37:27.220652 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c588d34a-e96d-4c50-a085-b234c15ca835 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:27.232092 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c588d34a-e96d-4c50-a085-b234c15ca835 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:27.236679 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b38f3-2199-4ec2-a17f-74d73982ace9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:27.237333 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b38f3-2199-4ec2-a17f-74d73982ace9 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:27.237618 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b38f3-2199-4ec2-a17f-74d73982ace9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:27.237892 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b38f3-2199-4ec2-a17f-74d73982ace9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:27.238608 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9b38f3-2199-4ec2-a17f-74d73982ace9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:27.240327 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9b38f3-2199-4ec2-a17f-74d73982ace9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:27.251086 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1430/2866] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:37:27.255257 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c10ec-f04b-4a37-a319-da5849a66e5e admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:27.265331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c10ec-f04b-4a37-a319-da5849a66e5e admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:27.269932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eab0b1-c9bf-4fe2-b059-8c4c6992cb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:27.270803 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eab0b1-c9bf-4fe2-b059-8c4c6992cb2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:27.271087 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eab0b1-c9bf-4fe2-b059-8c4c6992cb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:27.271400 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eab0b1-c9bf-4fe2-b059-8c4c6992cb2d admin admin] PATH_INFO: `/v3/projects/df92d9ac68ed479b85ea7b10f79fc979/users/6b4df19c89ef45158b241063e55653be/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:27.286565 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eab0b1-c9bf-4fe2-b059-8c4c6992cb2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=df92d9ac68ed479b85ea7b10f79fc979, user_id=6b4df19c89ef45158b241063e55653be, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:27.290058 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eab0b1-c9bf-4fe2-b059-8c4c6992cb2d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:27.302576 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1437/2867] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:37:27 2022] PUT /identity/v3/projects/df92d9ac68ed479b85ea7b10f79fc979/users/6b4df19c89ef45158b241063e55653be/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:27.306556 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b38f3-2199-4ec2-a17f-74d73982ace9 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:27.316312 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b38f3-2199-4ec2-a17f-74d73982ace9 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:27.328649 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:27.374328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6599d223-1468-4c35-9e79-bfcfba87105d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:27.374927 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599d223-1468-4c35-9e79-bfcfba87105d admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:27.375210 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599d223-1468-4c35-9e79-bfcfba87105d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:27.375519 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599d223-1468-4c35-9e79-bfcfba87105d admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:27.376238 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6599d223-1468-4c35-9e79-bfcfba87105d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:27.377826 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6599d223-1468-4c35-9e79-bfcfba87105d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:27.387524 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1431/2868] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:37:27.391847 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eab0b1-c9bf-4fe2-b059-8c4c6992cb2d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:27.402829 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eab0b1-c9bf-4fe2-b059-8c4c6992cb2d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:27.407486 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ec36a-8f6e-496d-a1e6-ff71d6df0a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:27.408361 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ec36a-8f6e-496d-a1e6-ff71d6df0a42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:27.408676 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ec36a-8f6e-496d-a1e6-ff71d6df0a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:27.408951 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ec36a-8f6e-496d-a1e6-ff71d6df0a42 admin admin] PATH_INFO: `/v3/projects/df92d9ac68ed479b85ea7b10f79fc979/users/6b4df19c89ef45158b241063e55653be/roles/82bbe3651b9b4f378fd150d93357b2b1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:27.410936 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ec36a-8f6e-496d-a1e6-ff71d6df0a42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df92d9ac68ed479b85ea7b10f79fc979, user_id=6b4df19c89ef45158b241063e55653be, role_id=82bbe3651b9b4f378fd150d93357b2b1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:27.414258 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ec36a-8f6e-496d-a1e6-ff71d6df0a42 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:27.424402 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1438/2869] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:37:27 2022] PUT /identity/v3/projects/df92d9ac68ed479b85ea7b10f79fc979/users/6b4df19c89ef45158b241063e55653be/roles/82bbe3651b9b4f378fd150d93357b2b1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:27.436158 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a14bb-5b41-4ba5-a2e2-919d8e3a1068 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:27.436694 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a14bb-5b41-4ba5-a2e2-919d8e3a1068 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:27.436694 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a14bb-5b41-4ba5-a2e2-919d8e3a1068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:27.464268 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-873a14bb-5b41-4ba5-a2e2-919d8e3a1068 None None] MFA Rules not processed for user `6b4df19c89ef45158b241063e55653be`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:27.465458 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-873a14bb-5b41-4ba5-a2e2-919d8e3a1068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:27.533746 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1432/2870] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:27.545067 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ec36a-8f6e-496d-a1e6-ff71d6df0a42 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:27.546781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e93ec36a-8f6e-496d-a1e6-ff71d6df0a42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:27.604275 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ec36a-8f6e-496d-a1e6-ff71d6df0a42 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:27.607953 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600b236-0688-435c-88a5-2812b0d99540 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:27.608580 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600b236-0688-435c-88a5-2812b0d99540 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:27.608865 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600b236-0688-435c-88a5-2812b0d99540 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:27.609141 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600b236-0688-435c-88a5-2812b0d99540 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:27.618085 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600b236-0688-435c-88a5-2812b0d99540 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:27.619577 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600b236-0688-435c-88a5-2812b0d99540 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:27.629004 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1439/2871] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:37:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:37:27.756771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a14bb-5b41-4ba5-a2e2-919d8e3a1068 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:27.817104 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a14bb-5b41-4ba5-a2e2-919d8e3a1068 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:27.822113 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc93e8-d4e7-45e4-a028-f34e3061db84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:27.822985 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc93e8-d4e7-45e4-a028-f34e3061db84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:27.823293 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc93e8-d4e7-45e4-a028-f34e3061db84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:27.823573 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc93e8-d4e7-45e4-a028-f34e3061db84 admin admin] PATH_INFO: `/v3/users/e9593be5392b45f58e416d4b689955fa` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:27.824747 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc93e8-d4e7-45e4-a028-f34e3061db84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9593be5392b45f58e416d4b689955fa)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:27.826655 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc93e8-d4e7-45e4-a028-f34e3061db84 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:27.866725 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:27.872669 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cebc93e8-d4e7-45e4-a028-f34e3061db84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9593be5392b45f58e416d4b689955fa'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:27.873570 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cebc93e8-d4e7-45e4-a028-f34e3061db84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9593be5392b45f58e416d4b689955fa'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:27.874289 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cebc93e8-d4e7-45e4-a028-f34e3061db84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9593be5392b45f58e416d4b689955fa'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:27.881054 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cebc93e8-d4e7-45e4-a028-f34e3061db84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9593be5392b45f58e416d4b689955fa'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:27.892933 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cebc93e8-d4e7-45e4-a028-f34e3061db84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9593be5392b45f58e416d4b689955fa'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:27.906883 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1433/2872] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:37:27 2022] DELETE /identity/v3/users/e9593be5392b45f58e416d4b689955fa => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:27.911464 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce9811-b41a-4c90-9680-fc4eba62c8db None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:27.911759 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce9811-b41a-4c90-9680-fc4eba62c8db None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:27.912039 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce9811-b41a-4c90-9680-fc4eba62c8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:27.929056 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-80ce9811-b41a-4c90-9680-fc4eba62c8db None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:27.930252 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-80ce9811-b41a-4c90-9680-fc4eba62c8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:27.973698 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:27.990691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1440/2873] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:28.001683 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc93e8-d4e7-45e4-a028-f34e3061db84 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:28.003096 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cebc93e8-d4e7-45e4-a028-f34e3061db84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:28.060390 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc93e8-d4e7-45e4-a028-f34e3061db84 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:28.064034 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287cca8c-ed98-4201-89dd-40587fe728df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:28.064671 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cca8c-ed98-4201-89dd-40587fe728df service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:28.064949 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cca8c-ed98-4201-89dd-40587fe728df service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:28.065219 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cca8c-ed98-4201-89dd-40587fe728df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:28.075447 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287cca8c-ed98-4201-89dd-40587fe728df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:28.076896 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287cca8c-ed98-4201-89dd-40587fe728df service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:28.086633 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1434/2874] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:37:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:37:28.357136 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce9811-b41a-4c90-9680-fc4eba62c8db None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:28.358632 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-80ce9811-b41a-4c90-9680-fc4eba62c8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:28.372782 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce9811-b41a-4c90-9680-fc4eba62c8db None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:28.377456 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae1f86-f9cb-47d4-bdc9-cdc036157b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:28.378312 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae1f86-f9cb-47d4-bdc9-cdc036157b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:28.378595 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae1f86-f9cb-47d4-bdc9-cdc036157b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:28.378882 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae1f86-f9cb-47d4-bdc9-cdc036157b3c admin admin] PATH_INFO: `/v3/projects/3ce8738e3b814eedacd33571ec02ff25` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:28.380094 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dae1f86-f9cb-47d4-bdc9-cdc036157b3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ce8738e3b814eedacd33571ec02ff25)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:28.382012 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dae1f86-f9cb-47d4-bdc9-cdc036157b3c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:28.428453 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3dae1f86-f9cb-47d4-bdc9-cdc036157b3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ce8738e3b814eedacd33571ec02ff25', 'request_id': 'req-3dae1f86-f9cb-47d4-bdc9-cdc036157b3c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:28.437149 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3dae1f86-f9cb-47d4-bdc9-cdc036157b3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ce8738e3b814eedacd33571ec02ff25', 'request_id': 'req-3dae1f86-f9cb-47d4-bdc9-cdc036157b3c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:28.443680 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3dae1f86-f9cb-47d4-bdc9-cdc036157b3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ce8738e3b814eedacd33571ec02ff25', 'request_id': 'req-3dae1f86-f9cb-47d4-bdc9-cdc036157b3c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:28.454787 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3dae1f86-f9cb-47d4-bdc9-cdc036157b3c admin admin] The token cache is being invalidate because project 3ce8738e3b814eedacd33571ec02ff25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:37:28.455076 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3dae1f86-f9cb-47d4-bdc9-cdc036157b3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:28.455743 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3dae1f86-f9cb-47d4-bdc9-cdc036157b3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:28.457315 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1441/2875] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:37:28 2022] DELETE /identity/v3/projects/3ce8738e3b814eedacd33571ec02ff25 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:28.461554 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287cca8c-ed98-4201-89dd-40587fe728df service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:28.471038 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287cca8c-ed98-4201-89dd-40587fe728df service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:28.528674 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75710d-3518-484e-8dad-1e2a95ef1542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:28.529547 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75710d-3518-484e-8dad-1e2a95ef1542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:28.529830 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75710d-3518-484e-8dad-1e2a95ef1542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:28.530105 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75710d-3518-484e-8dad-1e2a95ef1542 admin admin] PATH_INFO: `/v3/users/6b4df19c89ef45158b241063e55653be` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:28.531307 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd75710d-3518-484e-8dad-1e2a95ef1542 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b4df19c89ef45158b241063e55653be)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:28.533222 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd75710d-3518-484e-8dad-1e2a95ef1542 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:28.578678 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fd75710d-3518-484e-8dad-1e2a95ef1542 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b4df19c89ef45158b241063e55653be'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:28.579341 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fd75710d-3518-484e-8dad-1e2a95ef1542 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b4df19c89ef45158b241063e55653be'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:28.579931 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fd75710d-3518-484e-8dad-1e2a95ef1542 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b4df19c89ef45158b241063e55653be'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:28.588352 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fd75710d-3518-484e-8dad-1e2a95ef1542 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b4df19c89ef45158b241063e55653be'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:28.593506 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:28.603129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fd75710d-3518-484e-8dad-1e2a95ef1542 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b4df19c89ef45158b241063e55653be'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:28.615796 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1435/2876] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:37:28 2022] DELETE /identity/v3/users/6b4df19c89ef45158b241063e55653be => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:28.878223 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae1f86-f9cb-47d4-bdc9-cdc036157b3c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:28.879714 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3dae1f86-f9cb-47d4-bdc9-cdc036157b3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:28.892691 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae1f86-f9cb-47d4-bdc9-cdc036157b3c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:28.950424 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc069e21-d4b8-45b6-807d-b75a73d93a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:28.951305 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc069e21-d4b8-45b6-807d-b75a73d93a4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:28.951586 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc069e21-d4b8-45b6-807d-b75a73d93a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:28.951857 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc069e21-d4b8-45b6-807d-b75a73d93a4b admin admin] PATH_INFO: `/v3/projects/df92d9ac68ed479b85ea7b10f79fc979` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:28.953040 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc069e21-d4b8-45b6-807d-b75a73d93a4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=df92d9ac68ed479b85ea7b10f79fc979)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:28.954944 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc069e21-d4b8-45b6-807d-b75a73d93a4b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:29.003914 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cc069e21-d4b8-45b6-807d-b75a73d93a4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df92d9ac68ed479b85ea7b10f79fc979', 'request_id': 'req-cc069e21-d4b8-45b6-807d-b75a73d93a4b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:29.011579 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cc069e21-d4b8-45b6-807d-b75a73d93a4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df92d9ac68ed479b85ea7b10f79fc979', 'request_id': 'req-cc069e21-d4b8-45b6-807d-b75a73d93a4b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:29.017860 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cc069e21-d4b8-45b6-807d-b75a73d93a4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df92d9ac68ed479b85ea7b10f79fc979', 'request_id': 'req-cc069e21-d4b8-45b6-807d-b75a73d93a4b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:29.029394 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cc069e21-d4b8-45b6-807d-b75a73d93a4b admin admin] The token cache is being invalidate because project df92d9ac68ed479b85ea7b10f79fc979 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:37:29.029770 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cc069e21-d4b8-45b6-807d-b75a73d93a4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:29.030432 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cc069e21-d4b8-45b6-807d-b75a73d93a4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:29.032031 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1442/2877] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:37:28 2022] DELETE /identity/v3/projects/df92d9ac68ed479b85ea7b10f79fc979 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:29.036877 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b1a9d-3136-476e-92f7-a28bfadc936d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:29.037169 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b1a9d-3136-476e-92f7-a28bfadc936d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:29.037452 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b1a9d-3136-476e-92f7-a28bfadc936d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:29.055993 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1f3b1a9d-3136-476e-92f7-a28bfadc936d None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:29.057205 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1f3b1a9d-3136-476e-92f7-a28bfadc936d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:29.127946 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1436/2878] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:29.136689 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9e4e7-b40f-4461-b4e8-f8dcadd2103b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:29.136984 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9e4e7-b40f-4461-b4e8-f8dcadd2103b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:29.137292 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9e4e7-b40f-4461-b4e8-f8dcadd2103b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:29.143777 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:29.158659 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-def9e4e7-b40f-4461-b4e8-f8dcadd2103b None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:29.161145 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-def9e4e7-b40f-4461-b4e8-f8dcadd2103b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:29.170359 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1443/2879] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:29.174582 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b1a9d-3136-476e-92f7-a28bfadc936d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:29.184426 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b1a9d-3136-476e-92f7-a28bfadc936d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:29.188920 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf5b803-406e-4328-9ced-7b182481d25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:29.189572 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf5b803-406e-4328-9ced-7b182481d25a admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:29.189856 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf5b803-406e-4328-9ced-7b182481d25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:29.190133 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf5b803-406e-4328-9ced-7b182481d25a admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:29.190858 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf5b803-406e-4328-9ced-7b182481d25a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:37:29.191202 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf5b803-406e-4328-9ced-7b182481d25a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:29.192804 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf5b803-406e-4328-9ced-7b182481d25a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:29.204935 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1437/2880] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:37:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:37:29.209030 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9e4e7-b40f-4461-b4e8-f8dcadd2103b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:29.219108 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9e4e7-b40f-4461-b4e8-f8dcadd2103b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:29.223858 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7270446-1adb-49a5-aa53-94552bf0571f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:29.224501 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7270446-1adb-49a5-aa53-94552bf0571f admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:29.224784 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7270446-1adb-49a5-aa53-94552bf0571f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:29.225061 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7270446-1adb-49a5-aa53-94552bf0571f admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:29.226040 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7270446-1adb-49a5-aa53-94552bf0571f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:29.228040 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7270446-1adb-49a5-aa53-94552bf0571f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:29.252826 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1444/2881] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:37:29 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:37:29.256932 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf5b803-406e-4328-9ced-7b182481d25a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:29.267454 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf5b803-406e-4328-9ced-7b182481d25a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:29.323946 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c6a8b-105c-4e61-b2f7-059d1585ab99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:29.324509 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c6a8b-105c-4e61-b2f7-059d1585ab99 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:29.324974 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c6a8b-105c-4e61-b2f7-059d1585ab99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:29.325102 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c6a8b-105c-4e61-b2f7-059d1585ab99 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:29.325762 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c6a8b-105c-4e61-b2f7-059d1585ab99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:29.327866 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c6a8b-105c-4e61-b2f7-059d1585ab99 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:29.349233 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1438/2882] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:37:29 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:37:29.354203 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7270446-1adb-49a5-aa53-94552bf0571f admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:29.364957 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7270446-1adb-49a5-aa53-94552bf0571f admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:29.369680 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8e9dd-e3e8-4db4-ae7d-a405c827b8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:29.370340 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8e9dd-e3e8-4db4-ae7d-a405c827b8d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:29.370624 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8e9dd-e3e8-4db4-ae7d-a405c827b8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:29.370903 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8e9dd-e3e8-4db4-ae7d-a405c827b8d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:29.371667 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8e9dd-e3e8-4db4-ae7d-a405c827b8d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:29.373260 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8e9dd-e3e8-4db4-ae7d-a405c827b8d0 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:29.384170 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1445/2883] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:37:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:37:29.388129 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c6a8b-105c-4e61-b2f7-059d1585ab99 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:29.398727 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c6a8b-105c-4e61-b2f7-059d1585ab99 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:29.403375 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d607a9-c020-4620-8899-d1dc0438b955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:29.404040 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d607a9-c020-4620-8899-d1dc0438b955 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:29.404328 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d607a9-c020-4620-8899-d1dc0438b955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:29.404606 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d607a9-c020-4620-8899-d1dc0438b955 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:29.405337 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d607a9-c020-4620-8899-d1dc0438b955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:29.406920 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d607a9-c020-4620-8899-d1dc0438b955 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:29.417284 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1439/2884] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:37:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:37:29.421255 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8e9dd-e3e8-4db4-ae7d-a405c827b8d0 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:29.431807 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8e9dd-e3e8-4db4-ae7d-a405c827b8d0 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:29.436403 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ab27b-1085-43ef-bbf2-f4809cdbef6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:29.437266 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ab27b-1085-43ef-bbf2-f4809cdbef6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:29.437549 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ab27b-1085-43ef-bbf2-f4809cdbef6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:29.437824 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ab27b-1085-43ef-bbf2-f4809cdbef6d admin admin] PATH_INFO: `/v3/projects/a48b1f7432f94c6b98095086408f618b/users/1de00e9a8d4d4915b3f0219ab633a2c9/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:29.453916 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ab27b-1085-43ef-bbf2-f4809cdbef6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a48b1f7432f94c6b98095086408f618b, user_id=1de00e9a8d4d4915b3f0219ab633a2c9, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:29.456605 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ab27b-1085-43ef-bbf2-f4809cdbef6d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:29.471311 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1446/2885] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:37:29 2022] PUT /identity/v3/projects/a48b1f7432f94c6b98095086408f618b/users/1de00e9a8d4d4915b3f0219ab633a2c9/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:29.482774 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585d7c4-f81c-4acf-91dd-68548e109622 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:29.483319 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585d7c4-f81c-4acf-91dd-68548e109622 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:29.483428 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585d7c4-f81c-4acf-91dd-68548e109622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:29.511110 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1585d7c4-f81c-4acf-91dd-68548e109622 None None] MFA Rules not processed for user `1de00e9a8d4d4915b3f0219ab633a2c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:29.512280 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1585d7c4-f81c-4acf-91dd-68548e109622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:29.544321 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:29.577456 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1440/2886] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:29.589135 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ab27b-1085-43ef-bbf2-f4809cdbef6d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:29.590808 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0c9ab27b-1085-43ef-bbf2-f4809cdbef6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:29.669389 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ab27b-1085-43ef-bbf2-f4809cdbef6d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:29.673116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065a950-d673-4d51-af43-c7dcd5b6f5ba service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6180393da0f495db964f0b2588d8b90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:29.673791 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065a950-d673-4d51-af43-c7dcd5b6f5ba service glance] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:29.674072 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065a950-d673-4d51-af43-c7dcd5b6f5ba service glance] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:29.674347 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065a950-d673-4d51-af43-c7dcd5b6f5ba service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:29.683198 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065a950-d673-4d51-af43-c7dcd5b6f5ba service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:29.684650 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065a950-d673-4d51-af43-c7dcd5b6f5ba service glance] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:29.693388 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1447/2887] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:37:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:37:39.574609 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=11 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:39.689441 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=61,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:46.344610 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1585d7c4-f81c-4acf-91dd-68548e109622 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:46.407701 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1585d7c4-f81c-4acf-91dd-68548e109622 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:46.412196 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d09179-8eb7-4092-8cf5-dda3a10c42e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:46.413067 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d09179-8eb7-4092-8cf5-dda3a10c42e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:46.413351 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d09179-8eb7-4092-8cf5-dda3a10c42e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:46.413627 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d09179-8eb7-4092-8cf5-dda3a10c42e8 admin admin] PATH_INFO: `/v3/users/1de00e9a8d4d4915b3f0219ab633a2c9` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:46.414805 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d09179-8eb7-4092-8cf5-dda3a10c42e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1de00e9a8d4d4915b3f0219ab633a2c9)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:46.416788 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d09179-8eb7-4092-8cf5-dda3a10c42e8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:46.462188 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-00d09179-8eb7-4092-8cf5-dda3a10c42e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1de00e9a8d4d4915b3f0219ab633a2c9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:46.463416 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-00d09179-8eb7-4092-8cf5-dda3a10c42e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1de00e9a8d4d4915b3f0219ab633a2c9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:46.464168 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-00d09179-8eb7-4092-8cf5-dda3a10c42e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1de00e9a8d4d4915b3f0219ab633a2c9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:46.470771 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-00d09179-8eb7-4092-8cf5-dda3a10c42e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1de00e9a8d4d4915b3f0219ab633a2c9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:46.482520 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-00d09179-8eb7-4092-8cf5-dda3a10c42e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1de00e9a8d4d4915b3f0219ab633a2c9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:46.495334 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1441/2888] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:37:46 2022] DELETE /identity/v3/users/1de00e9a8d4d4915b3f0219ab633a2c9 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:46.499956 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f89aa-24df-4613-9395-62fd6fb3fdf9 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:46.500439 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f89aa-24df-4613-9395-62fd6fb3fdf9 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:46.500589 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f89aa-24df-4613-9395-62fd6fb3fdf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:46.518427 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-268f89aa-24df-4613-9395-62fd6fb3fdf9 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:46.519658 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-268f89aa-24df-4613-9395-62fd6fb3fdf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:46.578409 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1448/2889] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:46.588710 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d09179-8eb7-4092-8cf5-dda3a10c42e8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:46.590127 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-00d09179-8eb7-4092-8cf5-dda3a10c42e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:46.647979 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d09179-8eb7-4092-8cf5-dda3a10c42e8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:46.652468 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a2c07a-8411-4817-b5a4-b0497b0bac36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:46.653118 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2c07a-8411-4817-b5a4-b0497b0bac36 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:46.653408 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2c07a-8411-4817-b5a4-b0497b0bac36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:46.653694 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2c07a-8411-4817-b5a4-b0497b0bac36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:46.662713 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a2c07a-8411-4817-b5a4-b0497b0bac36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:46.664162 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a2c07a-8411-4817-b5a4-b0497b0bac36 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:46.673496 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1442/2890] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:37:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:37:46.940193 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f89aa-24df-4613-9395-62fd6fb3fdf9 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:46.941906 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-268f89aa-24df-4613-9395-62fd6fb3fdf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:46.954401 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f89aa-24df-4613-9395-62fd6fb3fdf9 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:46.958961 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadb91f-1922-4e60-967a-f2c0de25d9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:46.959903 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadb91f-1922-4e60-967a-f2c0de25d9a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:46.960181 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadb91f-1922-4e60-967a-f2c0de25d9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:46.960470 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadb91f-1922-4e60-967a-f2c0de25d9a1 admin admin] PATH_INFO: `/v3/projects/a48b1f7432f94c6b98095086408f618b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:46.961640 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbadb91f-1922-4e60-967a-f2c0de25d9a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a48b1f7432f94c6b98095086408f618b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:46.963996 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbadb91f-1922-4e60-967a-f2c0de25d9a1 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:47.011277 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-fbadb91f-1922-4e60-967a-f2c0de25d9a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a48b1f7432f94c6b98095086408f618b', 'request_id': 'req-fbadb91f-1922-4e60-967a-f2c0de25d9a1'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:47.018481 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-fbadb91f-1922-4e60-967a-f2c0de25d9a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a48b1f7432f94c6b98095086408f618b', 'request_id': 'req-fbadb91f-1922-4e60-967a-f2c0de25d9a1'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:47.024518 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-fbadb91f-1922-4e60-967a-f2c0de25d9a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a48b1f7432f94c6b98095086408f618b', 'request_id': 'req-fbadb91f-1922-4e60-967a-f2c0de25d9a1'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:47.034737 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-fbadb91f-1922-4e60-967a-f2c0de25d9a1 admin admin] The token cache is being invalidate because project a48b1f7432f94c6b98095086408f618b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:37:47.035019 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-fbadb91f-1922-4e60-967a-f2c0de25d9a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:47.035662 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-fbadb91f-1922-4e60-967a-f2c0de25d9a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:47.037186 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1449/2891] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:37:46 2022] DELETE /identity/v3/projects/a48b1f7432f94c6b98095086408f618b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:47.042743 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e6d83-bdcb-4e31-946d-7a463dc5f9c1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:47.043032 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e6d83-bdcb-4e31-946d-7a463dc5f9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:47.043334 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e6d83-bdcb-4e31-946d-7a463dc5f9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:47.061189 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-594e6d83-bdcb-4e31-946d-7a463dc5f9c1 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:47.062543 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-594e6d83-bdcb-4e31-946d-7a463dc5f9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:47.083045 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:47.130028 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1443/2892] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:47.138766 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412eef3-198e-483c-bb0a-7917ed07de67 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:47.139054 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412eef3-198e-483c-bb0a-7917ed07de67 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:47.139366 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412eef3-198e-483c-bb0a-7917ed07de67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:47.158365 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-c412eef3-198e-483c-bb0a-7917ed07de67 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:47.159582 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c412eef3-198e-483c-bb0a-7917ed07de67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:47.167818 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1450/2893] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:47.172002 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e6d83-bdcb-4e31-946d-7a463dc5f9c1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:47.181948 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e6d83-bdcb-4e31-946d-7a463dc5f9c1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:47.186556 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1caee-4cb8-4e59-b246-60e07042624d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:47.187232 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1caee-4cb8-4e59-b246-60e07042624d admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:47.187524 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1caee-4cb8-4e59-b246-60e07042624d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:47.187803 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1caee-4cb8-4e59-b246-60e07042624d admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:47.188815 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1caee-4cb8-4e59-b246-60e07042624d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:37:47.189144 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1caee-4cb8-4e59-b246-60e07042624d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:47.190766 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1caee-4cb8-4e59-b246-60e07042624d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:47.203172 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1444/2894] 10.222.0.81 () {58 vars in 1152 bytes} [Tue Jun 7 14:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:37:47.207389 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412eef3-198e-483c-bb0a-7917ed07de67 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:47.217316 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412eef3-198e-483c-bb0a-7917ed07de67 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:47.221915 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c28b0c-69f3-4063-9c5b-0bcd47262b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:47.222595 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c28b0c-69f3-4063-9c5b-0bcd47262b65 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:47.222869 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c28b0c-69f3-4063-9c5b-0bcd47262b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:47.223144 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c28b0c-69f3-4063-9c5b-0bcd47262b65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:47.223856 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c28b0c-69f3-4063-9c5b-0bcd47262b65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:47.225982 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c28b0c-69f3-4063-9c5b-0bcd47262b65 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:47.251439 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1451/2895] 10.222.0.81 () {60 vars in 1152 bytes} [Tue Jun 7 14:37:47 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:37:47.255549 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1caee-4cb8-4e59-b246-60e07042624d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:47.266165 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1caee-4cb8-4e59-b246-60e07042624d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:47.320884 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f428a-20b8-49d6-be4d-97ac12ad0495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:47.321483 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f428a-20b8-49d6-be4d-97ac12ad0495 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:47.322309 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f428a-20b8-49d6-be4d-97ac12ad0495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:47.322677 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f428a-20b8-49d6-be4d-97ac12ad0495 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:47.323434 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8f428a-20b8-49d6-be4d-97ac12ad0495 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:47.325384 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8f428a-20b8-49d6-be4d-97ac12ad0495 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:47.346267 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1445/2896] 10.222.0.81 () {60 vars in 1143 bytes} [Tue Jun 7 14:37:47 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:37:47.350331 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c28b0c-69f3-4063-9c5b-0bcd47262b65 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:47.361141 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c28b0c-69f3-4063-9c5b-0bcd47262b65 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:47.365945 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49217c69-1ded-4720-887f-0e9b7938a3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:47.366599 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49217c69-1ded-4720-887f-0e9b7938a3a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:47.366881 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49217c69-1ded-4720-887f-0e9b7938a3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:47.367152 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49217c69-1ded-4720-887f-0e9b7938a3a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:47.367900 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49217c69-1ded-4720-887f-0e9b7938a3a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:47.369588 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49217c69-1ded-4720-887f-0e9b7938a3a2 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:47.380892 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1452/2897] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:37:47.385320 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f428a-20b8-49d6-be4d-97ac12ad0495 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:47.395181 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f428a-20b8-49d6-be4d-97ac12ad0495 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:47.399789 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b3ce0-109f-465e-aee4-8994cf83770f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:47.401057 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b3ce0-109f-465e-aee4-8994cf83770f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:47.401340 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b3ce0-109f-465e-aee4-8994cf83770f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:47.401617 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b3ce0-109f-465e-aee4-8994cf83770f admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:47.402377 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b3ce0-109f-465e-aee4-8994cf83770f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:47.403983 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b3ce0-109f-465e-aee4-8994cf83770f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:47.413980 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1446/2898] 10.222.0.81 () {58 vars in 1121 bytes} [Tue Jun 7 14:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:37:47.417932 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49217c69-1ded-4720-887f-0e9b7938a3a2 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:47.427733 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49217c69-1ded-4720-887f-0e9b7938a3a2 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:47.432861 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff970717-d598-4d4a-9c7a-7477417738a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:47.433754 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff970717-d598-4d4a-9c7a-7477417738a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:47.434035 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff970717-d598-4d4a-9c7a-7477417738a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:47.434309 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff970717-d598-4d4a-9c7a-7477417738a9 admin admin] PATH_INFO: `/v3/projects/7690e258df704cb99b5bab3ff1c374f4/users/bcc74acac9234def9ce29e7f106658aa/roles/ca91f64e7a46431c8aa4285c4620f216` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:47.449272 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff970717-d598-4d4a-9c7a-7477417738a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7690e258df704cb99b5bab3ff1c374f4, user_id=bcc74acac9234def9ce29e7f106658aa, role_id=ca91f64e7a46431c8aa4285c4620f216)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:47.451939 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff970717-d598-4d4a-9c7a-7477417738a9 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:47.464767 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1453/2899] 10.222.0.81 () {60 vars in 1482 bytes} [Tue Jun 7 14:37:47 2022] PUT /identity/v3/projects/7690e258df704cb99b5bab3ff1c374f4/users/bcc74acac9234def9ce29e7f106658aa/roles/ca91f64e7a46431c8aa4285c4620f216 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:37:47.474975 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19744819-c205-4d51-8085-0cf9e2c3766d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:47.475295 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19744819-c205-4d51-8085-0cf9e2c3766d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:47.475579 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19744819-c205-4d51-8085-0cf9e2c3766d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:47.501875 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-19744819-c205-4d51-8085-0cf9e2c3766d None None] MFA Rules not processed for user `bcc74acac9234def9ce29e7f106658aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:47.503070 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-19744819-c205-4d51-8085-0cf9e2c3766d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:47.565428 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1447/2900] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:47.578317 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff970717-d598-4d4a-9c7a-7477417738a9 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:47.580116 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ff970717-d598-4d4a-9c7a-7477417738a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:47.602782 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:47.640216 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff970717-d598-4d4a-9c7a-7477417738a9 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:47.643900 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203cf50-b3ff-4ded-a20c-84029831c8c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b1ff21b8e5445b878740a327e4442f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:47.644542 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203cf50-b3ff-4ded-a20c-84029831c8c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:47.644827 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203cf50-b3ff-4ded-a20c-84029831c8c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:47.645105 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203cf50-b3ff-4ded-a20c-84029831c8c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:47.653781 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203cf50-b3ff-4ded-a20c-84029831c8c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:47.655206 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203cf50-b3ff-4ded-a20c-84029831c8c7 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:47.664058 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1454/2901] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:37:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:37:50.211473 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19744819-c205-4d51-8085-0cf9e2c3766d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:50.231320 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:50.276740 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19744819-c205-4d51-8085-0cf9e2c3766d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:50.281715 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29f74f-5103-4dbb-a404-98c9cfda27a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:50.282587 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29f74f-5103-4dbb-a404-98c9cfda27a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:50.282871 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29f74f-5103-4dbb-a404-98c9cfda27a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:50.283144 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29f74f-5103-4dbb-a404-98c9cfda27a2 admin admin] PATH_INFO: `/v3/users/bcc74acac9234def9ce29e7f106658aa` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:50.284363 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29f74f-5103-4dbb-a404-98c9cfda27a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcc74acac9234def9ce29e7f106658aa)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:50.286272 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29f74f-5103-4dbb-a404-98c9cfda27a2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:50.331304 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3f29f74f-5103-4dbb-a404-98c9cfda27a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcc74acac9234def9ce29e7f106658aa'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:50.331951 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3f29f74f-5103-4dbb-a404-98c9cfda27a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcc74acac9234def9ce29e7f106658aa'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:50.332545 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3f29f74f-5103-4dbb-a404-98c9cfda27a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcc74acac9234def9ce29e7f106658aa'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:50.339795 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3f29f74f-5103-4dbb-a404-98c9cfda27a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcc74acac9234def9ce29e7f106658aa'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:50.351497 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3f29f74f-5103-4dbb-a404-98c9cfda27a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcc74acac9234def9ce29e7f106658aa'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:50.363811 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1448/2902] 10.222.0.81 () {58 vars in 1223 bytes} [Tue Jun 7 14:37:50 2022] DELETE /identity/v3/users/bcc74acac9234def9ce29e7f106658aa => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:37:50.368553 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45b4f9-9139-431f-82fe-6ce3813bb588 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:50.369091 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45b4f9-9139-431f-82fe-6ce3813bb588 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:50.369412 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45b4f9-9139-431f-82fe-6ce3813bb588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:50.387350 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-da45b4f9-9139-431f-82fe-6ce3813bb588 None None] MFA Rules not processed for user `2604a49f226a4bdf958c4047caf193e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:50.388531 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-da45b4f9-9139-431f-82fe-6ce3813bb588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:50.446474 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1455/2903] 10.222.0.81 () {58 vars in 957 bytes} [Tue Jun 7 14:37:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:50.457180 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29f74f-5103-4dbb-a404-98c9cfda27a2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:50.458565 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3f29f74f-5103-4dbb-a404-98c9cfda27a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:50.515833 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29f74f-5103-4dbb-a404-98c9cfda27a2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:50.519442 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb12cea-a7bd-43f4-8394-f3002383c26b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a08eaecbcd41228571739d8387882a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:50.520087 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb12cea-a7bd-43f4-8394-f3002383c26b service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:50.520369 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb12cea-a7bd-43f4-8394-f3002383c26b service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:50.520644 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb12cea-a7bd-43f4-8394-f3002383c26b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:50.529801 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb12cea-a7bd-43f4-8394-f3002383c26b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:50.531238 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb12cea-a7bd-43f4-8394-f3002383c26b service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:50.540787 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1449/2904] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:37:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:37:50.811179 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45b4f9-9139-431f-82fe-6ce3813bb588 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:50.812600 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-da45b4f9-9139-431f-82fe-6ce3813bb588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:50.825815 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45b4f9-9139-431f-82fe-6ce3813bb588 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:50.830518 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42387242-f87b-4404-8442-421f20c02d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2604a49f226a4bdf958c4047caf193e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50e31e6b5bc94b188158e931e4ee4076', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:50.831417 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42387242-f87b-4404-8442-421f20c02d30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:50.831704 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42387242-f87b-4404-8442-421f20c02d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:50.831983 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42387242-f87b-4404-8442-421f20c02d30 admin admin] PATH_INFO: `/v3/projects/7690e258df704cb99b5bab3ff1c374f4` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:50.833332 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42387242-f87b-4404-8442-421f20c02d30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7690e258df704cb99b5bab3ff1c374f4)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:50.835311 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42387242-f87b-4404-8442-421f20c02d30 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:50.882698 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-42387242-f87b-4404-8442-421f20c02d30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7690e258df704cb99b5bab3ff1c374f4', 'request_id': 'req-42387242-f87b-4404-8442-421f20c02d30'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:50.890297 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-42387242-f87b-4404-8442-421f20c02d30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7690e258df704cb99b5bab3ff1c374f4', 'request_id': 'req-42387242-f87b-4404-8442-421f20c02d30'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:50.896736 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-42387242-f87b-4404-8442-421f20c02d30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7690e258df704cb99b5bab3ff1c374f4', 'request_id': 'req-42387242-f87b-4404-8442-421f20c02d30'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:50.907356 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-42387242-f87b-4404-8442-421f20c02d30 admin admin] The token cache is being invalidate because project 7690e258df704cb99b5bab3ff1c374f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:37:50.907643 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-42387242-f87b-4404-8442-421f20c02d30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:50.908279 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-42387242-f87b-4404-8442-421f20c02d30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:37:50.909823 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1456/2905] 10.222.0.81 () {58 vars in 1232 bytes} [Tue Jun 7 14:37:50 2022] DELETE /identity/v3/projects/7690e258df704cb99b5bab3ff1c374f4 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:38:00.536554 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:38:00.904341 dv-cinder-836083-9-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:39.362088 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb12cea-a7bd-43f4-8394-f3002383c26b service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:39.363554 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-eeb12cea-a7bd-43f4-8394-f3002383c26b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:39.432172 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb12cea-a7bd-43f4-8394-f3002383c26b service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:39.491275 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47099dc4-e836-46ad-a6f5-3426849f1eaa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab73cf3febbf4c768cd0116306f16c50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f67bad575f8b4a47bee7c314497cb2fe', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:39.491922 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47099dc4-e836-46ad-a6f5-3426849f1eaa service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:39.492207 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47099dc4-e836-46ad-a6f5-3426849f1eaa service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:39.492485 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47099dc4-e836-46ad-a6f5-3426849f1eaa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:39.494265 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-47099dc4-e836-46ad-a6f5-3426849f1eaa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:39.506737 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47099dc4-e836-46ad-a6f5-3426849f1eaa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:39.508342 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47099dc4-e836-46ad-a6f5-3426849f1eaa service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:39.518362 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1450/2906] 10.222.0.81 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:49.513879 dv-cinder-836083-9-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=58 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}